Final Report Summary - PACHELBEL (Policy addressing climate change and learning about consumer behavior and everyday life)
Executive summary:
The everyday consumer behaviour of citizens is increasingly recognized as an essential consideration in making policies aimed at promoting sustainability work in practice. Many existing policies have been focused on trying to promote 'more educated' consumer choices. Although citizens seem to recognise the seriousness of the problem, and the need to act in order to solve it, the available evidence suggests that the proposed environmental objectives are not being achieved.
Project Context and Objectives:
1.Introduction
This report provides a concise, non-technical, summary of project PACHELBEL. It describes the motivation behind the project, the approach adopted to address the various objectives, the ways in which the project was implemented in practice, and the main findings and outputs produced by the work. It also reports on the in-house evaluation of the project. Finally, it considers the potential for future work made possible by the findings to date, and the efforts currently being made to develop the work beyond the initial objectives.
We have concentrated on writing a full account of the project's work, but without straying into obscure or technical considerations. The reader should therefore not be surprised to find no references to the academic or technical literature, or footnotes that elaborate the text in terms of technical detail. Such detail is available in the deliverable reports produced by the project team, and in the numerous conference papers and technical publications that are either already available, or currently in preparation. Readers seeking such details should consult the project's website, or contact the Scientific Coordinator (see the contact details section).
In Section 2 (Project context and objectives), we outline the motivation for the project by considering the context in which we recognised what we regarded as a gap in the resources available to support policy-making for sustainability. We then set out how we proposed to address this gap in practical ways, and the objectives that we set for a proposed project. This research design served to attract European Union funding support.
In Section 3 (Main scientific and technical results), we describe the ways in which we implemented the project during the period January 2010-December 2012. This account includes our conceptual orientation, methodology, findings, and in-house evaluation. In Section 4 (Potential Impacts), we consider the impacts already achieved by the project, including practical support to work on live policy issues and various kinds of dissemination activity. We conclude this section by describing our efforts in further developing the achievements of the project, and we bring the report to a close with some conclusions. Section 5 (Contact details) sets out the organisational structure of the project, and provides contact details.
2.Project context and objectives
Project motivation and perspective
Citizen behaviour is increasingly recognized as an essential consideration in making policies aimed at promoting sustainability work in practice. Technological developments on their own, although essential, are simply not sufficient on their own to achieve sustainable environmental performance. In recent years multiple initiatives aimed at modifying consumer behaviour in order to enable and encourage more sustainable habits have been developed. Citizen responses to Eurobarometer surveys, for example, indicate high levels of concern about environmental issues like climate change (95%), and a willingness to modify their behaviours (87%) in an effort to mitigate such effects. On the basis of these citizen accounts, high levels of effectiveness might be expected from measures aiming to support behavioural changes designed to enhance sustainable consumption. However, there is considerable evidence that real-world citizen behaviour does not match these stated aspirations. Citizen energy consumption levels, for example, keep increasing, representing some 26% of the total energy consumption in the EU.
Although policy makers and citizens both apparently recognize the seriousness of the problem, and the need to act in order to solve it, the available evidence suggests that the proposed objectives are not being achieved. The European Commission's latest evaluation of the EU Energy Efficiency Plan, for example, shows that only 20% of the 2020 objectives have been achieved. The significant challenge, and relative intractability of, attempts to shift citizen consumption behaviour in more sustainable directions has now been recognised by policymakers and scholars alike, and the practical difficulties entailed in this objective are increasingly clear.
In order to address these difficulties, policy-makers have begun to recognize that shifting citizen behaviour amounts to far more than a matter of simply informing consumers about the impact of their behaviours. In other words, there is more to it than explaining 'the facts' and expecting them to behave 'sensibly'. A number of strands of research suggest that such behaviours are embedded within a matrix of factors like everyday associations, preferred ways of life, economic constraints, and emotional commitments. In recognition of this complexity, policy initiatives now tend to take a more sophisticated form than being simply about 'public education', and include elements of communication, advertising, incentives, and citizen engagement.
The need for a means to support policymaking for sustainability by exploring the lived practicalities of everyday environment-related citizen behaviours is evident. This was the driving motivation behind project PACHELBEL.
Research approach
In order to address these difficulties, we proposed to take as our point of departure the increasingly large body of research and practical findings concerned with citizen engagement. The trend towards citizen engagement as a policy tool had developed significantly within many democratic countries over the previous decade. The use of various forms of extended consultation, participation and deliberative involvement with members of the lay public has been advocated as a means to address a number of perceived difficulties of contemporary governance, including deficits of knowledge, trust and legitimacy.
Of great significance has been a recognition of the limits of the extent to which technical knowledge alone provides a suitable basis for the resolution of many decision-making questions concerned with innovation and technology management. In cases where the issue in question is associated with some degree of controversy, and conflicts exist in underlying values and motivations, difficulties arise in seeking to reconcile expert knowledge and the needs of the market with strongly held beliefs and commitments within affected communities. Of particular interest for the design of project PACHELBEL has been recent work which has used hybrid group-based methods, which combine processes of research and engagement, in order to investigate lay citizen's practical reasoning about technology decision-making.
We recognised that any serious attempt to find ways of enhancing the utilisation of research knowledge in policymaking needed to have regard to the concrete nature of the policymaking process within specific organisational settings. Whilst there exists a vast research literature on organisations and organisational decision-making, the volume of work on the role of knowledge in organisational settings is much more limited; with a few sources being especially significant. These sources recognise that knowledge cannot be viewed as a 'package' which can be transferred within organisations in unproblematic ways. Rather, organisational processes entail situated processes of gathering, presenting, disputing and agreeing, practical reasoning about, and finally using different forms and sources of knowledge. In practical terms, these findings suggested the need for us to gain a fairly intimate degree of insight into the everyday practicalities of the policymaking process in order to embed in effective ways the policymaking support tool (STAVE) that we proposed to develop. This consideration led us to propose a problem-focused (or action research) approach, in which we would work closely with policymaking organisations in order to support their work in addressing live policy issues.
So the key methodological theme running through the preliminary design for project PACHELBEL was one of engagement. We proposed to engage with the policymaking process by means of collaborative action research interventions with policy partners. We planned to implement a process of engagement with lay citizens, by means of research/deliberation/engagement processes. The latter would tap into features of the everyday lives of lay groups, their practical reasoning and learning processes, and the likely impact on their lives of a range of sustainability-related policy initiatives. Further stages of the work would establish linkages between these two processes of engagement, which would complete the interconnected set of practices constituting the STAVE tool.
According to our design, the entire trajectory of developing, trialling and operationalising STAVE would include an in-built process of evaluation.
Project objectives
We identified a series of objectives for the project. The primary objectives would be concerned with the production of the STAVE tool. We appreciated that the engagement processes entailed in the development of STAVE would provide us with in-depth research access to naturalistic aspects of both the policymaking process, and to citizens' everyday sustainability-related behaviours. These latter considerations formed the basis for setting out the secondary objectives.
Primary objectives
-To develop, trial and operationalise the STAVE tool in the context of concrete policy-related interventions.
-To implement a systematic evaluation of the development and performance of the STAVE tool.
-To produce guidance on how best to use the STAVE tool across a range of policy environments.
-To fully disseminate the findings of the project.
Secondary objectives
-To investigate policy-making practices and processes concerned with addressing anthropogenic climate change and sustainable consumption, with particular regard to the ways in which assumptions about human behaviour are incorporated into these practices.
-To investigate the nature of practical reasoning by lay citizens about anthropogenic climate change and sustainable consumption, with particular regard to its relationship with their everyday consumer behaviours.
-To produce a range of scholarly publications, conference presentations and other outputs.
Project Results:
3. Methodology and findings
3.1 Conceptual orientation
Our points of departure were two plausible - as we saw them - assumptions:
First, it seemed clear to us that policies seeking to promote sustainable consumption would be more effective if they made workable assumptions about the likely impacts of those policies on citizen behaviour. For us, this meant that empirical evidence needed to be at the heart of the project.
Second, it seemed reasonable to suppose that the practical reasoning associated with policymaking for sustainability, on the one hand, and the practical reasoning deployed by citizens in the conduct of domestic and other aspects of everyday life (which may have implications for sustainability), on the other, are distinct in nature. This is not to say that either policymakers or citizens going about their everyday business are 'irrational' with respect to the other. Rather, these activities simply entail different ways of looking at the world; different forms of life, one might say.
Importantly, agreement on these basic assumptions allowed the PACHELBEL research team - a group of individuals with diverse disciplinary backgrounds - to work together in a cohesive fashion, and in ways that largely avoided the sort of conceptual and methodological disputes that are only too common within the social sciences and in attempts at multi-disciplinary collaboration. In this way, by shifting the focus of the work away from theorisation towards engagement with practice, and the collection of rich evidence, PACHELBEL has emerged as a coherent, transdisciplinary and problem-focused project.
It followed from these assumptions that we needed to develop a tool that would support policymakers by making visible the nature of citizens' culturally-shaped behaviours that were, in some technical sense, related to sustainability, and how those behaviours were rationalised by citizens. It also followed that in order to make that tool work effectively, we needed to find a way to bridge these two different worlds - to allow them to communicate intelligibly with each other.
Preliminary informal conversations with policymakers had indicated that such a tool, especially if was relatively speedy and inexpensive in application, would have considerable potential utility within often messy, contested and fast-moving policy environments. It was likely to be helpful, we felt, across a wide range of policy-related activity: from strategic policymaking to local implementation.
We began the implementation of the project by 'going back to basics' in this way. Therefore, in some sense, we began the process of designing and building STAVE from scratch.
3.2 Methods: conceiving and implementing the STAVE tool
Tackling the methodological challenge
In finding a means to investigate citizens' everyday sustainability-related behaviours, the straightforward notion of 'going and asking them what they do' first presented itself, suggesting the possible use of some kind of survey or interview approach. However, this option immediately raised the issue of the nature of the accounts generated by different research instruments, and the potential gap between 'what they say' and 'what they do'; in other words between the accounts that people provide of their behaviours or intended behaviours when responding to interviews or surveys, and the measurable impact of their actual behaviours.
Much of the sustainability literature discusses, and attempts to map or measure citizen attitudes about environmental issues, and the potential for citizens to adopt sustainable consumption practices. We regard this activity as inherently problematic. There is a sense in which such investigations seek to elicit some unequivocal piece of knowledge which is located 'in the heads' of individual respondents. In response, we would suggest that this approach would be most likely to elicit accounts which are appropriate responses to a researcher who poses a question framed in terms of technical environmental considerations. In so doing, we would argue, the researcher decontextualises the issue of sustainability from everyday domestic life. In this way, a gap has been created between the practical reasoning deployed by lay citizens in going about their everyday domestic lives and an account generated in an abstracted context.
How can one interpret this gap between actual and claimed behaviours? Perhaps people fail to understand the wider significance of their everyday practices. Possibly the answers they provide reflect intentions or aspirations; how they would liked to have behaved. After all, being seen to be 'not environmentally friendly' might plausibly be regarded as socially unacceptable, across an increasingly large proportion of cultures, classes and societies. In this way, the answers provided to a survey or interview can reflect not only the technical framing of the question, but also a tendency for respondents to present themselves 'in a good light'.
Of course, this is not to say that accounts generated by interviews or surveys will bear no relation to the everyday practices to which they allude. In general, there will be some correspondence between accounts and actions. However, it is not clear how one might determine the nature of that correspondence. Here one can see the roots of the gap between 'what they say' and 'what they do'. In some sense, there is the possibility that respondents providing answers to research questions might be said to be 'playing a different game' from when they are engaged in getting on with the mundane tasks entailed in their everyday domestic lives.
One possible way of side-stepping this question of the situated nature of accounts might be to construct a tool based primarily on observing citizen sustainability-related behaviours in naturally-occurring settings. Some research has already pursued this sort of approach by means of setting up monitoring equipment to capture actual behaviours in real time. Of course, aside from the inherent technical difficulties and potential cost, the problem with this approach is that it does not necessarily provide insights into the meanings of the practices in question for those enacting the practices. If one wishes to design more effective policies to promote sustainable consumption then understanding citizen behaviours and motivations would appear to be important considerations.
These considerations about citizen accounts suggested to us the need to adopt some kind of ethnographic method that would allow us to capture naturalistic citizen behaviours. But the workability and cost of such an approach seemed to present an insurmountable obstacle. How could we make progress?
The use of small groups
We turned to small group-based methods. Focus groups have the potential to generate particular sorts of conversation that can reveal socially-shared ways of reasoning and acting among groups of people with shared experiences and ways of life. This property makes possible the use of focus groups as a valid research method. It also provides a means to gain access to features of social life for which research in naturally-occurring settings is difficult or unworkably expensive.
We drew upon extensive work using focus groups that had been carried out by members of the team. This work had included a series of experiments in recent years of hybridising focus groups with other methods, including participant diaries, and methods drawn from citizen engagement practice and Operational Research/Management Science. Such hybridisation had attempted to promote enhanced participant engagement, so generating accounts of the world that were grounded in everyday practices to a greater degree than was sometimes possible with conventional focus groups. These experiments had provided a powerful means to research citizen reasoning about issues as diverse as railway safety and nuclear fusion power, and the ways in which that reasoning can change as people learn more about the technicalities and social context of such technologies. The new hybrid methods had demonstrated their capacity to allow citizens to reason about technical and complex issues in familiar terms.
An important part of the research literature has analysed the sorts of group conversations that are needed to get different sorts of work done. Such kinds of conversation range from relatively unstructured occasions, like going shopping with friends, to those that require more in the way of structure and technique, like business meetings, to highly-formalised and structured processes like those present in courts of law. We knew from our experience with hybrid focus groups that the use of some techniques was effective in promoting conversations that were good at promoting certain kinds of activity, for example: exploring patterns of activity in domestic kitchens, or discussing difficult choices within the family budget. But our knowledge was far from complete. We needed to learn a great deal about the properties of different sorts of group conversation, and how to promote them, before we could use that design knowledge to specify the STAVE tool.
We concluded that our prototype STAVE tool would need to be trialled in different sorts of real-world situation in order to explore the practicalities of what sorts of group conversation were most effective in providing suitable policymaking support.
Linking and embedding the STAVE tool
Citizens would be recruited to take part in group conversations about aspects of their everyday lives that related to the policy issue in question. But how, in practical terms, could we find ways of implementing the arrows that represent lines of intelligible communication?
We needed to translate questions posed by the policymakers in ways that would make sense to the citizen group conversations. We also needed to find ways of translating what the citizens had to say in ways that were of relevance to the work of policymaking. The latter challenge meant that we needed to find out a great deal about the social world of policymaking, and how to embed the STAVE tool within that world.
In line with our commitment to trialling the prototype STAVE tool in real-world policy situations, and to an action research approach to the work, we approached a range of public policy organisations across the six European countries from which the PACHELBEL partners were drawn. These organisations had responsibility for the development and/or implementation of policy relating to environmental sustainability. They included strategic organisations forming part of national governments, and local administrations, located at regional or city levels. We proposed collaborative arrangements in which we would provide policymaking support for one or more live policy issues. This work - which might be described as quasi-consultancy - would be provided free of charge in exchange for research access to the everyday work of the policy organisation, and co-operation in trialling STAVE.
We were successful in establishing collaborative arrangements with policy organisations in all six countries. Now we had gained access to these organisations, we needed to learn a great deal about their work, with a view to embedding the STAVE trialling process. In the same way as we planned to build an ethnographic dimension into STAVE, so as to gain access to naturalistic citizen behaviours, so we wished to appreciate naturalistic aspects of how policy work was done by our collaborating organisations. Ethnography is, of course, potentially expensive in terms of researcher time commitment. Also it was not clear whether all the collaborating organisations would have welcomed us being an almost daily presence in their work places over an extended period.
In practice, only one of the project partners was able to carry out such a comprehensive ethnographic study. This was made possible by the partner in question securing additional funding for a student to carry out the ethnography as the central part of their doctoral studies. The rest of the partners adopted a variety of fieldwork practices that involved them gaining as much access to the everyday work of their collaborating organisation as possible, and at the same time, building good working relationships with their staff. Importantly, the style of these fieldwork investigations - including attending planning meetings, discussion of internal documentation, informal visits, formal and informal interviews, telephone calls and emails - were characterised by an ethnographic style and sensibility. A uniform and high-quality approach to this organisational fieldwork was promoted by the monthly circulation among the team of brief reports on findings, and feedback to partners provided by the project coordinator and team methodologist.
Building the prototype STAVE tool
On the basis of our existing experience with group-based methods, we assembled a set of techniques - a provisional collection of what we came to call the tool-kit - which we knew tended to promote different sorts of conversation. In this way, we began to create a tool that could be used in a targeted way to elicit data from citizen groups. We trialled these techniques in a series of STAVE interventions in which we attempted to explore and capture different aspects of citizens' actual behaviours in ways that allowed us to gather rich behavioural data relating to the policy issue(s) in question.
Component parts of the tool-kit included: the use of oval maps as a visual representation of group ideas, which in turn provided a flexible cumulative record of the discussions; a set of stimulus materials, including simulated news items and cartoons; diaries to link everyday practices with the group conversations; vignettes; a resource allocation task; and questionnaires to elicit personal identification with aspects of sustainable consumption. With these techniques we attempted to generate discussions that served to make visible understandings and practices that were socially shared by participant citizens. The short sustainability-related questionnaire devices were used to elicit 'in principle' accounts of behaviours, which sometimes contrasted in insightful ways with the more grounded patterns of shared practical reasoning evident in the group discussions, and in the diaries of daily activities that participants completed.
During the trialling exercises, each group (comprising 8-10 citizens) met on three occasions for around 90 minutes. Participants were asked keep a simple diary during the intervening periods between group meetings.
During the project's trialling phase, 157 people took part in 18 STAVE interventions. To enable comparative analyses of STAVE performance, we agreed that national trials would be encouraged to follow the same general approach to implementing group discussions. A degree of flexibility was necessary to allow individual teams to adapt their interventions to their specific national circumstances and needs. A consequence of this combined approach of comparability and flexibility is that the implementation of STAVE components varied slightly between countries, with partners developing specific instruments tailored to their national setting.
The organisation of the project work
PACHELBEL was organised according to a matrix pattern. In this way, partner organisations divided their time commitment between STAVE trialling work in their own countries, and project-wide work package responsibilities. Six work packages were identified: overall coordination and administration; the overall methodological integration and the in-house evaluation of the project; the investigation of the work of collaborating policy organisations; the development of stimulus material for the group process; the design of the group process; and the dissemination of project findings and outputs. Nine of the ten partners worked to implemented STAVE trials within their six countries. Seven of the principal researchers adopted lead roles of the six work package functions. The project scientific coordinator provided overall leadership, supported by her own organisation team, and by the project methodologist. Specific details of who did what is set out at section 4 (Potential Impacts).
3.3 Primary objectives: the STAVE trialling process and capturing STAVE design knowledge
Reiteration of the primary objectives
The project was concerned primarily with the design, development and operationalisation of the STAVE policymaking support tool. This process necessitated trialling of the tool prototype in the context of real-world policy issues. The project and the STAVE tool were both to be evaluated in a rigorous fashion by means in in-house processes. All these objectives are considered in this part of the report.
The other primary objectives: to produce guidance on how best to use STAVE across a range of policy environments, and to disseminate the findings of the project, are both considered in Part 4 of this report, as they are concerned with existing or potential impacts of the project.
The trialling process
At an early stage in the project, we took the decision that STAVE needed to be trialled in the context of live, real-world, policy contexts, rather than simulated ones. We took the view that only by engaging with naturally-occurring policy situations would we be able to gain suitable 'buy-in' from policymakers and citizens, and so gain intimate access to their everyday patterns of behaviour and practical reasoning.
As an exercise in action research, our trialling process was driven primarily by the policymakers' concerns and needs, rather than by matters of scholarly interest. Such research necessitates the active involvement of practitioners throughout the research activity. It also often requires the work to be done according to timescales that are sometimes unusually short in comparison with that of much academic work. As such, action research can present academics with something of a challenge. Indeed, previous such research within policy settings has often been in the context of collaborations between policymakers and practitioner communities. In this case, the research team's considerable experience in applied and policy-related research made this potential difficulty less of a problem than it might have been for some pure researchers. The PACHELBEL team's focus on working closely with policymakers, and using STAVE to deliver practical support for policymaking in real time, proved successful in by far most of the collaborations. In this way, the trialling of the STAVE tool was grounded within the everyday social worlds of policymaking and everyday citizen practices.
In each country, we worked with our collaborating policymakers to identify a live policy issue where they had an active interest in exploring or validating their knowledge about citizen's sustainability-related behaviours. These specific policy issues were explored with specially-recruited groups of citizens using the STAVE methodology.
Data generated during the STAVE interventions provided feedback to our policymaker partners. We sought to make this feedback both meaningful and constructive. We did so by providing two kinds of feedback: the first kind, 'speedy', which drew directly on the citizen participants' deliberations, using a variety of 'low tech' resources (e.g. oval maps, 'brainstorming' exercises, data from the resource allocation exercises etc.) generated by the group itself; and second, a more orthodox data analysis of the group conversations and activities, which was conducted by the research team after the group process was completed. This orthodox data analysis played an important role in validating and fine-tuning the 'speedy' mechanisms.
It is important to note that the policy organisations with which we worked presented a range of contrasting features, including: their experience of conducting or commissioning social research; their familiarity with various kinds of citizen engagement; and the extent to which they had previously considered in explicit ways the diversity of citizen behaviours and practices.
In assessing the wider value of STAVE as a policymaking support tool it is important to consider the contexts in which the application of STAVE took place. The policy organisations in Germany, Sweden, UK and in one of the Spanish interventions had most experience of having commissioned social research into public views about sustainability issues. The UK organisation had probably commissioned the most substantial research work on lay views about climate change. In contrast, the policy organisation from Romania was less familiar with social research, and also had less direct experience of how to interpret data generated by public engagement.
In France, two of the policy organisations had roles that routinely made use of the results from social science research. They regarded themselves as 'end users' of social science research, but did not see themselves as having an appropriate role in producing or commissioning it. The third French organisation had a more direct experience of citizen engagement. In Spain, the organisation involved in Agenda 21 was very familiar with participatory approaches, and most of the officials had backgrounds in environmental education. The second Spanish organisation recognized the potential value of social research to support policymaking, but had no practical experience in this regard. Thus, there were considerable variations in the policy organisations' backgrounds and levels of experience, within as well as between countries.
The policy area most thoroughly addressed by the STAVE trials was the topic of energy use. This was addressed in a number of ways: patterns of spending and saving electricity; heat and hot water; the use of smart meters and their relationship with electricity savings; and domestic insulation. Energy topics were the focus for STAVE work in France, Germany, Romania and Spain. Spain and Sweden both considered mobility issues. In the UK, the focus was on consumers' understanding of product lifetimes and its relationship with shopping behaviour.
Significantly, we found evidence that policymakers' involvement in the use of STAVE promoted an enhanced degree of reflection on their part on the ways in which they considered policy issues, and on their existing knowledge and assumptions about citizen behaviours. During the close and dynamic collaboration between the policymakers and members of the PACHELBEL team, a number of the policymakers indicated that they had come to reflect on their own work, as well as on evidence gaps regarding consumer behaviour that STAVE trials served to access. As a result of the collaborative engagement, some policymakers, for example in Spain, decided to change the focus of their choice for STAVE intervention after having realised that their real interest was on shopkeepers' behaviour rather than on business premises. The processes of dialogue involved in selecting a suitable policy issue for STAVE intervention appears to have prompted policymakers to evaluate more closely the evidence base on what policies work and why. In turn, these processes served to surface implicit, or ill-formed, assumptions about the scope for encouraging shifts in citizen behaviour.
Turning to the citizen discussion and diary process, the evidence is clear that STAVE is a method that is capable of generating a high degree of constructive engagement with groups of citizens. Such engagement is highly effective in eliciting patterns of socially-shared everyday behaviours, and authentic ways of talking about such behaviours. At first the citizen participants were cautious, but as they become more involved in the process they clearly displayed a sense of freedom to explain their informal doubts and contradictions; features of their everyday experience that we suspect would be difficult to capture using more conventional social research methods.
As noted above, we tested two modes of feedback; one which we termed 'speedy' and the other 'orthodox'. The first one presented policymakers with materials generated directly by participants during the group process. The second one took the form of classical report, generated through more structured analysis of data generated by the group process. In most cases, both stages of feedback took place during face-to-face meetings we had with policymakers. Our presentation of feedback provided policymakers with the opportunity to seek additional information and clarification as required.
Capturing STAVE design knowledge
Having carried out the STAVE trials, we then needed to analyse the considerable volume of data collected. Examination of this body of evidence allowed us to move towards a systematic understanding of the properties of the prototype STAVE tool in generating data corresponding to a range of concrete policy issues. In this way, we developed design knowledge that would allow us to use the STAVE tool in a targeted way to support policymaking for specific policy issues. The resulting design knowledge was assembled in the form of the STAVE Brochure and STAVE Manual, which are described in more detail in Part 4 below.
3.4 Secondary objectives: the social worlds of policymaking and everyday citizen behaviours
Reiteration of the secondary objectives
The project's secondary objectives sought to make use of the research access to naturalistic aspects of policymaking and citizen behaviours that were necessary in order to satisfy the primary, STAVE tool-related, objectives. In order to develop and trial the STAVE tool, we needed to gain fairly intimate access to the social worlds of policymaking and everyday consumer practices. Having successfully addressed our primary objectives, which were of a practical, policy-orientated, nature, we were able to re-examine our data with the more research-oriented secondary objectives in mind.
Our secondary objectives were concerned with investigating the nature of policymaking practices and processes concerned with sustainable consumption, with particular regard to the ways in which assumptions about human behaviour are incorporated into these practices. We were also concerned with the nature of practical reasoning by lay citizens about sustainable consumption, with particular regard to its relationship with their everyday consumer behaviours.
These secondary objectives were also concerned with the production of a range of scholarly publications, conference presentations and other outputs concerned with knowledge about these naturalistic aspects of policymaker and citizen behaviours. We will address the dissemination aspects of the secondary objectives in Part 4 below.
STAVE feedback and policy assumptions
The STAVE tool has shown its capacity to question assumptions that policymakers may have held about citizen behaviours. Most collaborating policymakers reacted to STAVE feedback with considerable interest, and sometimes surprise, as STAVE seemed to be able to 'bring alive' citizens' practices in a particularly lively and vivid way. The tool's capacity to show the real-world nature of the gap between discourses and practices of sustainability, and to be appreciated as such by policymakers, has been perhaps the most significant achievement of the project.
However, in a sense, the acronym STAVE ('Systematic Tool for Behavioural Assumption Validation and Exploration') that we adopted for the policymaking support tool we developed was something of a misnomer. The process of confronting policymakers with STAVE feedback has served to cast some doubt on whether clear assumptions about citizen behaviour were being incorporated into policy in the first place. Rather, it seems that a range of sometimes provisional and informal ideas - 'hunches' one might say - were being used in the process of policymaking. In itself, this is an important finding about the nature of policymaking practice.
In the cases where the policymaking organisations had commissioned research of their own, the ideas in use about citizen behaviours were more explicit and elaborated. Even in these circumstances, the ideas were often provisional and contested. Importantly, STAVE demonstrated its capacity to prompt policymaker reflections on what they knew and what they didn't know. The engendering of such processes of self-reflection seems to be an important property of STAVE as a policymaking support tool.
The practical accomplishment of policymaking
In working closely with a range of policymaking organisations, we were able to gain insights into the way that the everyday work of policymaking was done, in the real world of practice. These insights proved invaluable when it came to agreeing the nature of the STAVE interventions that we staged in order to support the specific policy issues with which our collaborators were engaged. It also helped us in seeking to present STAVE feedback in as helpful a way as possible. As we anticipated, in addition to helping us run the STAVE trials, these insights into the nature of policymaking work offered the possibility of significant research findings on the practical accomplishment of policymaking.
Although there exists a huge literature on the nature and work of policymaking organisations, very little of this material has been devoted to investigating naturalistic aspects of everyday policy work. As indicated earlier in this report, we recognised that capturing the dynamics of this work would necessitate a high degree of ethnographic immersion in the locations where the policy work was done. This degree of access was not possible in most of the organisations with which we worked. However, very significant volumes of such ethnographic data were collected in one location, with the potential to generate novel findings on the work of policymaking. The extent to which the intimate, but less intensive, data collection within the other collaborating organisations has the capacity to generate research findings is currently being explored during the preparation of publications.
Citizens' everyday sustainability-related practices
As discussed above, the development of the STAVE tool provided us with access to naturalistic aspects of the work of policymaking. It also provided access to naturalistic aspects of everyday citizen behaviours. The use of the STAVE tool-kit in different ways provided opportunities to investigate discourses of sustainability, the practices and accountability of citizen consumer practices, and the possibility of mismatch between formal commitments to sustainability and practical barriers to acting in sustainable ways. A significant volume of data was collected from the STAVE trials which allowed us to assemble comparative descriptions of citizen reasoning processes, as they related to: talking about and making sense of sustainability; living sustainably; and the gap between formal commitments or aspirations and real behaviours.
We found that in general, most participants in the STAVE groups displayed awareness that individual consumption practices concerning energy use, mobility and waste are strongly connected with sustainablility in the wider sense of environmental protection, climate change, nature conservation or responsibility for future generations. However, we found a great deal of evidence for a mismatch between framing consumption practices in terms of wider environmental issues or in terms of everyday domestic practices.
This mismatch constitutes a crucial methodological issue when seeking to investigate practical assumptions about citizen behaviours. Importantly, it provides strong empirical evidence to support our fundamental assumption about the gap between practical reasoning appropriate for policymaking for sustainability and practical reasoning for the conduct of everyday domestic life. These two spheres of human experience do indeed appear to constitute different forms of life. These dimensions of citizen sustainability-related experience will be explored in publications currently in preparation.
3.5 Evaluating the project and the STAVE tool
The in-house evaluation process
Throughout the course of the project, an in-house programme of evaluation was carried out. This activity began collecting data on the first day of the project and continued until the last day of the allotted lifetime of the work. The framework for evaluation drew upon technical aspects of evaluation theory and practice, and included measurement of performance in terms of process and outputs. The process measures drew upon recent evaluation research developments concerning the efficiency and effectiveness of the collection of data, and the flows of information between different stages of the work. The evaluation activity was designed to make extensive use of datasets that would be collected primarily as part of the central research and development activity, rather than specifically for evaluation. In this way, we set out to make the evaluation activity as unobtrusive and efficient as possible.
The roles of external interaction in evaluation: collaborative work, dissemination, international advisory committee and end-of-project workshops
In addition to the use of measures of internal effectiveness and efficiency, our interaction with the external world provided important opportunities to evaluate project performance. The successful development of the STAVE tool and its trialling in the context of live policy issues constitutes a powerful measure of the overall effectiveness of the project. The feedback received from collaborating policymakers was predominantly positive. In this way, we can point to firm evidence of the sound performance of the project.
A second external measure of effectiveness was provided by the interest shown by the scholarly community in our work. During the second year of the project, we proposed to the organisers of a major European conference that we present a collection of papers on our work in progress, and this was agreed and successfully executed. During the final year of the project, our work constituted dedicated symposia forming part of a major European conference, and a major international conference held in the United States. Feedback received from these occasions has been consistently positive. Our activity in disseminating the work and findings of the project at scholarly conference and other occasions will now form the basis for a series of peer-reviewed publications, which are currently in preparation. Members of our international advisory group have provided consistently supportive, but not uncritical, voices during the course of the work. Their capacity to force us to confront difficult issues, especially concerned with the overall framing and positioning of the work, has proved invaluable, and we are most grateful for their efforts.
Turning finally to the end-of-project workshops, we are delighted to be able to report, once again, on positive evaluations, delivered in such a way so as to fine-tune and enhance the quality of the project. In the main workshop, held in Brussels in September 2012, we examined the findings of the project with collaborating policy organisations and a number of officials from the European Commission. At the heart of the discussion were a series of joint presentations on the application of the STAVE tool to specific policy issues, presented by members of the project team with collaborating policymakers. Members of the international advisory committee joined in the discussions online. During the following month, we ran another workshop in Bucharest; this time focused on disseminating the project findings in Eastern Europe. Once again, we received positive feedback on what we had to offer.
The project's overall performance
Overall, the performance of project PACHELBEL has been highly satisfactory. It met all its objectives, including the design and operationalisation of the STAVE tool, and the trialling of that tool in the context of live policy settings. The feedback from our collaborating policy organisations has been strikingly positive. We feel that we are close to finalising STAVE as a potentially valuable contribution to the world of policymaking for sustainability.
A lively and authentic impression of the collaborating policy maker's enthusiasm for STAVE could already be gained at the 3rd Consortium Meeting (Stockholm) were some of them reported on STAVE:
-It did the job
-Addresses right topics
-Fosters a better understanding of needs of target groups
-Creates new ideas, approaches
-Delivers insights in values, norms, attitudes
-Good for ranking the value of different policy initiatives
-Delivers valuable and resilient ('robust') data for further internal consultation
-Flexible across different policy areas
-Very applicable in terms of costs and timescales
-Meaningful results in acceptable period of time
-Good at communicating questions to participants
PACHELBEL was essentially an action research project, in the sense that it was focused primarily on the achievement of practical tasks defined by the policy organisations with which we collaborated, rather than on objectives defined in terms of scholarly considerations. All such projects are pivotally dependent upon the establishment of satisfactory working relationships with collaborating bodies, which provide access to naturally-occurring work situations. Crucially, such research is also subject to the vagaries of the working environments in which the collaborating organisations operate. Given these considerations, it is especially important to stress the considerable success achieved by the project. The project team was able to implement STAVE trials across six different European countries, at different levels in public administration ranging from strategic to local, with a high degree of synchronisation, effectiveness, and compatibility.
We have a number of additional specific conclusions:
-STAVE is a user-friendly support tool that generates rich, grounded, data about everyday citizen behaviours in a relatively speedy way. It is a very flexible method, and can potentially be applied across a wide range of policy contexts.
-Despite being deceptively simple in design, STAVE requires deployment by facilitators with mature social research skills. Careful design work is needed to adapt the STAVE tool-kit to the specific research context. Mature social research capabilities are also needed in order to analyse the diverse data gathered, and to draw reliable conclusions.
-Different components of the STAVE toolkit seem to have different capacities to capture everyday features of sustainability-related citizen behaviours. The data generated by these different tools also varies in terms of its appeal to policymakers. It is perhaps the juxtaposition of diaries and face-to-face groups that provides one of the most distinctive elements of STAVE: its capacity to engender reflection on the relationship between 'in-principle' claims and grounded accounts of everyday practices.
-STAVE operates by brokering indigenous knowledges between the social worlds of policymaking and citizens' domestic and other everyday behaviours. In so doing, in creates intelligible communication between these distinct spheres of experience, action and rationality.
-In functioning as a policymaking support tool, STAVE also serves to promote critical reflection by policymakers of their existing knowledge, hunches and assumptions about what citizens do in their everyday lives. In this way, STAVE is able to enrich the policymaking process, by encouraging engagement by policymakers with authentic and grounded features of naturally-occurring citizen practices and rationalities.
-STAVE is not an orthodox citizen engagement tool. It is a policymaking support tool that produces forms of engagement with both citizen and policymakers. Policymaker involvement promotes self-refection about their own practices, and engenders sensitivity to citizens' cultural practices. As such, STAVE might be regarded as an effective means to generate capacity-building for both the policy community and civil society.
Potential Impact:
4. Existing and potential impacts
4.1 Contributions to addressing live policy issues: STAVE as a policy support tool
What worked well and what didn't work so well, and why
As already noted, the STAVE feedback that we generated for the policymakers was regarded by them as mostly useful and constructive. Most of the policy organisations involved in the project use social research in their activities in some way, usually to inform the design of policies to promote sustainability. In so doing, they have tended to set a clear boundary between their activities and the social research they commission or accessed in some other way. Significantly, the nature of STAVE interventions necessitates the active involvement of policymakers from the beginning of the process: from the formulation of the questions to be investigated; during the citizen group process, when they are invited to comment on preliminary feedback and perhaps refine their questions; to the final face-to-face engagement with the STAVE operators when the materials generated by the citizen groups are considered. For some of our policymaker collaborators, this degree of involvement in a research process seemed a little unusual. Yet, following their participation, most policymakers recognised the significant value of their involvement.
Most of the policymakers responded with interest, and sometimes surprise, when confronted with STAVE feedback. They mostly valued the opportunity to gain deeper insight into citizen behaviours, and the ways in which citizen rationalised those behaviours. In some cases the data confirmed their hunches and expectations. A key benefit, as recognised by policymakers in several countries, was the novel capacity of STAVE to shed some light on the problem we have already characterised as 'the gap between what citizens say and what they do'. A number of the policymakers were also pleasantly surprised by the capacity of STAVE to engender a willingness by citizen participants to talk openly about their domestic and other everyday practices. On occasions they were surprised by citizens' capacity to recognize, and reflect upon, the occurrence of inconsistencies between their environment-related practices and their accounts of those practices. Overall, a number of the policymakers were especially positive about how the immediate and unrefined feedback served to capture relevant evidence from citizens, and to make this available for the policymaking work in a relatively speedy, yet meaningful way.
The impact of STAVE
Turning now to how policymakers might use the STAVE findings, we have to acknowledge that the picture is more confused. It is not entirely clear how the STAVE interventions will, in the short term, influence policymaking and implementation concerning our target policy issues. Early comments from the collaborating policymakers have provided us with some encouragement about the potential future application and influence of STAVE. However, the eventual outcome seems likely to turn on the resolution of processes within the policy organisations. Of central importance, it seems, will be two questions that a number of the policymakers asked themselves: 'can this be used as an argument to be put forward to politicians? 'and 'does this help to determine where to concentrate our efforts? '
The perceived status of the STAVE findings in terms of how 'scientific' they might be regarded seems to be an important factor here. Generally, those more strategic policy organisations with which we worked found the early, 'immediate and unrefined', feedback less useful than the 'orthodox' feedback. Some regarded this material as very different in nature from the social research reports with which they were familiar. Importantly, the early feedback, in its somewhat raw and rich form, did not look like the fruits of standard social research. It is possible that such material was therefore not regarded as 'scientific', in some sense, and therefore could not be taken to politicians or senior decision-makers as evidence to justify action, especially if such action necessitated significant investment in terms of time and/or money. In contrast, the more locally-based policy organisations tended to find more immediate value in STAVE. Indeed, in places there was a real appetite to apply the findings, and to explore the applicability of STAVE in other policy areas.
It is clear from the evidence of the trials that there are limits to the extent to which STAVE is already able to provide what a number of policymakers would regard as a valuable support tool. However, it is important not to lose sight of the very significant positive messages that have emerged from the trials. The difficulties, such as they are, seem to be primarily concerned with finding ways of successfully embedding STAVE within the pre-existing patterns of practice and culture within policy organisations. We are currently seeking to address this obstacle by means of the preparations of materials that will allow prospective users to quickly understand the nature of the STAVE tool, and the ways in which it might provide specific policymaking support.
4.2 The STAVE Brochure and Manual
In line with our objective to produce guidance on how best to use the STAVE tool across a range of policy environments, we have used the STAVE design knowledge to prepare a STAVE Brochure and a STAVE Manual. The Brochure is a simple introduction to the STAVE tool, written with potential users in mind. The Manual is a more comprehensive resource prepared for potential users, which draws on case study material, and so supports realistic planning to use STAVE in specific policy contexts.
Both Brochure and Manual have been posted on the PACHELBEL website. Their initial format is as textual material, with links to supplementary visual and audio material. We plan to continue to enrich these resources, with our objective to produce multi-media web objects which include links to video materials.
4.3 Dissemination activities: project website, newsletter, conference presentations and publications
The team has been energetic in disseminating its work by means of a range of mechanisms. A PACHELBEL website has been maintained throughout the life of the project, containing up-to-date information about the project itself, and its events, findings and publications. The website continues to be maintained. Six issues of an illustrated electronic newsletter were produced and distributed widely. A poster and flyer were also produced as promotional materials. Members of the team contributed various talks, briefing and presentations to local, national and international events, including 18 papers to international conferences.
Some peer-reviewed papers have already started to appear, for example in the journal Periodistica, online in the series produced by the Spanish Communication Research Association, and in the working paper series published by the Swedish National defence College. Some are receiving minor revisions in the light of reviewer comments, for example a chapter from the book Knowledge Brokerage for a Sustainable Europe, and from the European Journal of Decision Processes. Others are currently under review or about to be submitted to journals like Energy Policy, Public Understanding of Science, Environmental Values, Environment and Behaviour, and Human Relations.
4.4 Next steps
Despite the success of project PACHELBEL, which we have described in this report, there exist a number of outstanding tasks which still require attention before STAVE can begin to play a significant role in policymaking for sustainability in Europe and beyond. At present, we are working on how best to put in place mechanisms that will allow these additional objectives to be satisfied.
Although a great deal of effort has already gone into disseminating the ideas, findings and practice of the project, establishing STAVE as well-known and well-regarded policymaking support tool is a potentially difficult and time-consuming process. STAVE has the potential to be useful across a wide range of policymaking contexts. Many policy organisations which could make use of STAVE may be difficult to contact, and they will almost certainly have a preference for what they regard as tried-and-tested ways of doing their work. They will also tend to only adopt new ways of doing things on recommendation from trusted contacts.
There is a potential circularity here. As the number of successful STAVE applications grows, and these can be documented and made available via the website, talks and publications, so the possibility of a more widespread use of STAVE will be become more likely. However, clearly developing this body of work necessitates the existence of wider access to policy organisations. Importantly, a number of policy organisations involved in PACHELBEL are playing a key role in this regard, as they are introducing the tool to other departments within their own organization.
Further developing the PACHELBEL website will be an important activity. As noted above, we plan to further enhance the STAVE Brochure and STAVE Good Practice Guide, by transforming them from primarily text-based resources into multi-media web objects, including audio and video material on our existing case studies. We are also investigating ways of making STAVE even more user-friendly, and to more effectively embed the tool within the work of policy organisations. Towards these ends, we are seeking new project work that will provide us with access to more naturally-occurring policy settings. Finally, we are exploring the means to establish an enduring network of STAVE users (STAVE Association), and a reservoir of case study-based STAVE application evidence. A first step in this regard is the creation of an STAVE Association, already being promoted by the project coordinators.
Conclusions
STAVE produces creative forms of engagement with both citizens and policymakers. Policymaker involvement in the STAVE process promotes self-refection about their own practices and their existing knowledge about citizen behaviours, and engenders sensitivity to citizens' indigenous practices. STAVE makes visible the nature and rationalities of citizens' environment-related practices by allowing citizens to discuss policy-related issues in their own everyday terms. Importantly, the tool can reveal the nature of practical barriers preventing the adoption of environmental-friendly consumer behaviours. The tool offers a means to allow policymakers to design and communicate their sustainability policies much more effectively.
Importantly:
-The use of STAVE is relatively inexpensive.
-STAVE is relatively speedy in use.
-STAVE is user-friendly for both citizens and policymakers.
-The feedback received from the collaborating policy organisations gaining support from STAVE has been strikingly positive.
Overall, the performance of project PACHELBEL has been highly satisfactory. It met all its objectives, including the design and operationalisation of the STAVE tool, and the trialling of that tool in the context of live policy settings.
PACHELBEL was essentially an action research project, in the sense that it was focused primarily on the achievement of practical tasks defined by the policy organisations with which we collaborated, rather than on objectives defined in terms of scholarly considerations. All such projects are pivotally dependent upon the establishment of satisfactory working relationships with collaborating bodies, which provide access to naturally-occurring work situations. Crucially, such research is also subject to the vagaries of the working environments in which the collaborating organisations operate. Given these considerations, it is especially important to stress the considerable success achieved by the project. The project team was able to implement STAVE trials across six different European countries, at different levels in public administration ranging from strategic to local, with a high degree of synchronisation, effectiveness, and compatibility. This is an impressive achievement.
List of Websites:
http://www.pachelbel.eu
The everyday consumer behaviour of citizens is increasingly recognized as an essential consideration in making policies aimed at promoting sustainability work in practice. Many existing policies have been focused on trying to promote 'more educated' consumer choices. Although citizens seem to recognise the seriousness of the problem, and the need to act in order to solve it, the available evidence suggests that the proposed environmental objectives are not being achieved.
Project Context and Objectives:
1.Introduction
This report provides a concise, non-technical, summary of project PACHELBEL. It describes the motivation behind the project, the approach adopted to address the various objectives, the ways in which the project was implemented in practice, and the main findings and outputs produced by the work. It also reports on the in-house evaluation of the project. Finally, it considers the potential for future work made possible by the findings to date, and the efforts currently being made to develop the work beyond the initial objectives.
We have concentrated on writing a full account of the project's work, but without straying into obscure or technical considerations. The reader should therefore not be surprised to find no references to the academic or technical literature, or footnotes that elaborate the text in terms of technical detail. Such detail is available in the deliverable reports produced by the project team, and in the numerous conference papers and technical publications that are either already available, or currently in preparation. Readers seeking such details should consult the project's website, or contact the Scientific Coordinator (see the contact details section).
In Section 2 (Project context and objectives), we outline the motivation for the project by considering the context in which we recognised what we regarded as a gap in the resources available to support policy-making for sustainability. We then set out how we proposed to address this gap in practical ways, and the objectives that we set for a proposed project. This research design served to attract European Union funding support.
In Section 3 (Main scientific and technical results), we describe the ways in which we implemented the project during the period January 2010-December 2012. This account includes our conceptual orientation, methodology, findings, and in-house evaluation. In Section 4 (Potential Impacts), we consider the impacts already achieved by the project, including practical support to work on live policy issues and various kinds of dissemination activity. We conclude this section by describing our efforts in further developing the achievements of the project, and we bring the report to a close with some conclusions. Section 5 (Contact details) sets out the organisational structure of the project, and provides contact details.
2.Project context and objectives
Project motivation and perspective
Citizen behaviour is increasingly recognized as an essential consideration in making policies aimed at promoting sustainability work in practice. Technological developments on their own, although essential, are simply not sufficient on their own to achieve sustainable environmental performance. In recent years multiple initiatives aimed at modifying consumer behaviour in order to enable and encourage more sustainable habits have been developed. Citizen responses to Eurobarometer surveys, for example, indicate high levels of concern about environmental issues like climate change (95%), and a willingness to modify their behaviours (87%) in an effort to mitigate such effects. On the basis of these citizen accounts, high levels of effectiveness might be expected from measures aiming to support behavioural changes designed to enhance sustainable consumption. However, there is considerable evidence that real-world citizen behaviour does not match these stated aspirations. Citizen energy consumption levels, for example, keep increasing, representing some 26% of the total energy consumption in the EU.
Although policy makers and citizens both apparently recognize the seriousness of the problem, and the need to act in order to solve it, the available evidence suggests that the proposed objectives are not being achieved. The European Commission's latest evaluation of the EU Energy Efficiency Plan, for example, shows that only 20% of the 2020 objectives have been achieved. The significant challenge, and relative intractability of, attempts to shift citizen consumption behaviour in more sustainable directions has now been recognised by policymakers and scholars alike, and the practical difficulties entailed in this objective are increasingly clear.
In order to address these difficulties, policy-makers have begun to recognize that shifting citizen behaviour amounts to far more than a matter of simply informing consumers about the impact of their behaviours. In other words, there is more to it than explaining 'the facts' and expecting them to behave 'sensibly'. A number of strands of research suggest that such behaviours are embedded within a matrix of factors like everyday associations, preferred ways of life, economic constraints, and emotional commitments. In recognition of this complexity, policy initiatives now tend to take a more sophisticated form than being simply about 'public education', and include elements of communication, advertising, incentives, and citizen engagement.
The need for a means to support policymaking for sustainability by exploring the lived practicalities of everyday environment-related citizen behaviours is evident. This was the driving motivation behind project PACHELBEL.
Research approach
In order to address these difficulties, we proposed to take as our point of departure the increasingly large body of research and practical findings concerned with citizen engagement. The trend towards citizen engagement as a policy tool had developed significantly within many democratic countries over the previous decade. The use of various forms of extended consultation, participation and deliberative involvement with members of the lay public has been advocated as a means to address a number of perceived difficulties of contemporary governance, including deficits of knowledge, trust and legitimacy.
Of great significance has been a recognition of the limits of the extent to which technical knowledge alone provides a suitable basis for the resolution of many decision-making questions concerned with innovation and technology management. In cases where the issue in question is associated with some degree of controversy, and conflicts exist in underlying values and motivations, difficulties arise in seeking to reconcile expert knowledge and the needs of the market with strongly held beliefs and commitments within affected communities. Of particular interest for the design of project PACHELBEL has been recent work which has used hybrid group-based methods, which combine processes of research and engagement, in order to investigate lay citizen's practical reasoning about technology decision-making.
We recognised that any serious attempt to find ways of enhancing the utilisation of research knowledge in policymaking needed to have regard to the concrete nature of the policymaking process within specific organisational settings. Whilst there exists a vast research literature on organisations and organisational decision-making, the volume of work on the role of knowledge in organisational settings is much more limited; with a few sources being especially significant. These sources recognise that knowledge cannot be viewed as a 'package' which can be transferred within organisations in unproblematic ways. Rather, organisational processes entail situated processes of gathering, presenting, disputing and agreeing, practical reasoning about, and finally using different forms and sources of knowledge. In practical terms, these findings suggested the need for us to gain a fairly intimate degree of insight into the everyday practicalities of the policymaking process in order to embed in effective ways the policymaking support tool (STAVE) that we proposed to develop. This consideration led us to propose a problem-focused (or action research) approach, in which we would work closely with policymaking organisations in order to support their work in addressing live policy issues.
So the key methodological theme running through the preliminary design for project PACHELBEL was one of engagement. We proposed to engage with the policymaking process by means of collaborative action research interventions with policy partners. We planned to implement a process of engagement with lay citizens, by means of research/deliberation/engagement processes. The latter would tap into features of the everyday lives of lay groups, their practical reasoning and learning processes, and the likely impact on their lives of a range of sustainability-related policy initiatives. Further stages of the work would establish linkages between these two processes of engagement, which would complete the interconnected set of practices constituting the STAVE tool.
According to our design, the entire trajectory of developing, trialling and operationalising STAVE would include an in-built process of evaluation.
Project objectives
We identified a series of objectives for the project. The primary objectives would be concerned with the production of the STAVE tool. We appreciated that the engagement processes entailed in the development of STAVE would provide us with in-depth research access to naturalistic aspects of both the policymaking process, and to citizens' everyday sustainability-related behaviours. These latter considerations formed the basis for setting out the secondary objectives.
Primary objectives
-To develop, trial and operationalise the STAVE tool in the context of concrete policy-related interventions.
-To implement a systematic evaluation of the development and performance of the STAVE tool.
-To produce guidance on how best to use the STAVE tool across a range of policy environments.
-To fully disseminate the findings of the project.
Secondary objectives
-To investigate policy-making practices and processes concerned with addressing anthropogenic climate change and sustainable consumption, with particular regard to the ways in which assumptions about human behaviour are incorporated into these practices.
-To investigate the nature of practical reasoning by lay citizens about anthropogenic climate change and sustainable consumption, with particular regard to its relationship with their everyday consumer behaviours.
-To produce a range of scholarly publications, conference presentations and other outputs.
Project Results:
3. Methodology and findings
3.1 Conceptual orientation
Our points of departure were two plausible - as we saw them - assumptions:
First, it seemed clear to us that policies seeking to promote sustainable consumption would be more effective if they made workable assumptions about the likely impacts of those policies on citizen behaviour. For us, this meant that empirical evidence needed to be at the heart of the project.
Second, it seemed reasonable to suppose that the practical reasoning associated with policymaking for sustainability, on the one hand, and the practical reasoning deployed by citizens in the conduct of domestic and other aspects of everyday life (which may have implications for sustainability), on the other, are distinct in nature. This is not to say that either policymakers or citizens going about their everyday business are 'irrational' with respect to the other. Rather, these activities simply entail different ways of looking at the world; different forms of life, one might say.
Importantly, agreement on these basic assumptions allowed the PACHELBEL research team - a group of individuals with diverse disciplinary backgrounds - to work together in a cohesive fashion, and in ways that largely avoided the sort of conceptual and methodological disputes that are only too common within the social sciences and in attempts at multi-disciplinary collaboration. In this way, by shifting the focus of the work away from theorisation towards engagement with practice, and the collection of rich evidence, PACHELBEL has emerged as a coherent, transdisciplinary and problem-focused project.
It followed from these assumptions that we needed to develop a tool that would support policymakers by making visible the nature of citizens' culturally-shaped behaviours that were, in some technical sense, related to sustainability, and how those behaviours were rationalised by citizens. It also followed that in order to make that tool work effectively, we needed to find a way to bridge these two different worlds - to allow them to communicate intelligibly with each other.
Preliminary informal conversations with policymakers had indicated that such a tool, especially if was relatively speedy and inexpensive in application, would have considerable potential utility within often messy, contested and fast-moving policy environments. It was likely to be helpful, we felt, across a wide range of policy-related activity: from strategic policymaking to local implementation.
We began the implementation of the project by 'going back to basics' in this way. Therefore, in some sense, we began the process of designing and building STAVE from scratch.
3.2 Methods: conceiving and implementing the STAVE tool
Tackling the methodological challenge
In finding a means to investigate citizens' everyday sustainability-related behaviours, the straightforward notion of 'going and asking them what they do' first presented itself, suggesting the possible use of some kind of survey or interview approach. However, this option immediately raised the issue of the nature of the accounts generated by different research instruments, and the potential gap between 'what they say' and 'what they do'; in other words between the accounts that people provide of their behaviours or intended behaviours when responding to interviews or surveys, and the measurable impact of their actual behaviours.
Much of the sustainability literature discusses, and attempts to map or measure citizen attitudes about environmental issues, and the potential for citizens to adopt sustainable consumption practices. We regard this activity as inherently problematic. There is a sense in which such investigations seek to elicit some unequivocal piece of knowledge which is located 'in the heads' of individual respondents. In response, we would suggest that this approach would be most likely to elicit accounts which are appropriate responses to a researcher who poses a question framed in terms of technical environmental considerations. In so doing, we would argue, the researcher decontextualises the issue of sustainability from everyday domestic life. In this way, a gap has been created between the practical reasoning deployed by lay citizens in going about their everyday domestic lives and an account generated in an abstracted context.
How can one interpret this gap between actual and claimed behaviours? Perhaps people fail to understand the wider significance of their everyday practices. Possibly the answers they provide reflect intentions or aspirations; how they would liked to have behaved. After all, being seen to be 'not environmentally friendly' might plausibly be regarded as socially unacceptable, across an increasingly large proportion of cultures, classes and societies. In this way, the answers provided to a survey or interview can reflect not only the technical framing of the question, but also a tendency for respondents to present themselves 'in a good light'.
Of course, this is not to say that accounts generated by interviews or surveys will bear no relation to the everyday practices to which they allude. In general, there will be some correspondence between accounts and actions. However, it is not clear how one might determine the nature of that correspondence. Here one can see the roots of the gap between 'what they say' and 'what they do'. In some sense, there is the possibility that respondents providing answers to research questions might be said to be 'playing a different game' from when they are engaged in getting on with the mundane tasks entailed in their everyday domestic lives.
One possible way of side-stepping this question of the situated nature of accounts might be to construct a tool based primarily on observing citizen sustainability-related behaviours in naturally-occurring settings. Some research has already pursued this sort of approach by means of setting up monitoring equipment to capture actual behaviours in real time. Of course, aside from the inherent technical difficulties and potential cost, the problem with this approach is that it does not necessarily provide insights into the meanings of the practices in question for those enacting the practices. If one wishes to design more effective policies to promote sustainable consumption then understanding citizen behaviours and motivations would appear to be important considerations.
These considerations about citizen accounts suggested to us the need to adopt some kind of ethnographic method that would allow us to capture naturalistic citizen behaviours. But the workability and cost of such an approach seemed to present an insurmountable obstacle. How could we make progress?
The use of small groups
We turned to small group-based methods. Focus groups have the potential to generate particular sorts of conversation that can reveal socially-shared ways of reasoning and acting among groups of people with shared experiences and ways of life. This property makes possible the use of focus groups as a valid research method. It also provides a means to gain access to features of social life for which research in naturally-occurring settings is difficult or unworkably expensive.
We drew upon extensive work using focus groups that had been carried out by members of the team. This work had included a series of experiments in recent years of hybridising focus groups with other methods, including participant diaries, and methods drawn from citizen engagement practice and Operational Research/Management Science. Such hybridisation had attempted to promote enhanced participant engagement, so generating accounts of the world that were grounded in everyday practices to a greater degree than was sometimes possible with conventional focus groups. These experiments had provided a powerful means to research citizen reasoning about issues as diverse as railway safety and nuclear fusion power, and the ways in which that reasoning can change as people learn more about the technicalities and social context of such technologies. The new hybrid methods had demonstrated their capacity to allow citizens to reason about technical and complex issues in familiar terms.
An important part of the research literature has analysed the sorts of group conversations that are needed to get different sorts of work done. Such kinds of conversation range from relatively unstructured occasions, like going shopping with friends, to those that require more in the way of structure and technique, like business meetings, to highly-formalised and structured processes like those present in courts of law. We knew from our experience with hybrid focus groups that the use of some techniques was effective in promoting conversations that were good at promoting certain kinds of activity, for example: exploring patterns of activity in domestic kitchens, or discussing difficult choices within the family budget. But our knowledge was far from complete. We needed to learn a great deal about the properties of different sorts of group conversation, and how to promote them, before we could use that design knowledge to specify the STAVE tool.
We concluded that our prototype STAVE tool would need to be trialled in different sorts of real-world situation in order to explore the practicalities of what sorts of group conversation were most effective in providing suitable policymaking support.
Linking and embedding the STAVE tool
Citizens would be recruited to take part in group conversations about aspects of their everyday lives that related to the policy issue in question. But how, in practical terms, could we find ways of implementing the arrows that represent lines of intelligible communication?
We needed to translate questions posed by the policymakers in ways that would make sense to the citizen group conversations. We also needed to find ways of translating what the citizens had to say in ways that were of relevance to the work of policymaking. The latter challenge meant that we needed to find out a great deal about the social world of policymaking, and how to embed the STAVE tool within that world.
In line with our commitment to trialling the prototype STAVE tool in real-world policy situations, and to an action research approach to the work, we approached a range of public policy organisations across the six European countries from which the PACHELBEL partners were drawn. These organisations had responsibility for the development and/or implementation of policy relating to environmental sustainability. They included strategic organisations forming part of national governments, and local administrations, located at regional or city levels. We proposed collaborative arrangements in which we would provide policymaking support for one or more live policy issues. This work - which might be described as quasi-consultancy - would be provided free of charge in exchange for research access to the everyday work of the policy organisation, and co-operation in trialling STAVE.
We were successful in establishing collaborative arrangements with policy organisations in all six countries. Now we had gained access to these organisations, we needed to learn a great deal about their work, with a view to embedding the STAVE trialling process. In the same way as we planned to build an ethnographic dimension into STAVE, so as to gain access to naturalistic citizen behaviours, so we wished to appreciate naturalistic aspects of how policy work was done by our collaborating organisations. Ethnography is, of course, potentially expensive in terms of researcher time commitment. Also it was not clear whether all the collaborating organisations would have welcomed us being an almost daily presence in their work places over an extended period.
In practice, only one of the project partners was able to carry out such a comprehensive ethnographic study. This was made possible by the partner in question securing additional funding for a student to carry out the ethnography as the central part of their doctoral studies. The rest of the partners adopted a variety of fieldwork practices that involved them gaining as much access to the everyday work of their collaborating organisation as possible, and at the same time, building good working relationships with their staff. Importantly, the style of these fieldwork investigations - including attending planning meetings, discussion of internal documentation, informal visits, formal and informal interviews, telephone calls and emails - were characterised by an ethnographic style and sensibility. A uniform and high-quality approach to this organisational fieldwork was promoted by the monthly circulation among the team of brief reports on findings, and feedback to partners provided by the project coordinator and team methodologist.
Building the prototype STAVE tool
On the basis of our existing experience with group-based methods, we assembled a set of techniques - a provisional collection of what we came to call the tool-kit - which we knew tended to promote different sorts of conversation. In this way, we began to create a tool that could be used in a targeted way to elicit data from citizen groups. We trialled these techniques in a series of STAVE interventions in which we attempted to explore and capture different aspects of citizens' actual behaviours in ways that allowed us to gather rich behavioural data relating to the policy issue(s) in question.
Component parts of the tool-kit included: the use of oval maps as a visual representation of group ideas, which in turn provided a flexible cumulative record of the discussions; a set of stimulus materials, including simulated news items and cartoons; diaries to link everyday practices with the group conversations; vignettes; a resource allocation task; and questionnaires to elicit personal identification with aspects of sustainable consumption. With these techniques we attempted to generate discussions that served to make visible understandings and practices that were socially shared by participant citizens. The short sustainability-related questionnaire devices were used to elicit 'in principle' accounts of behaviours, which sometimes contrasted in insightful ways with the more grounded patterns of shared practical reasoning evident in the group discussions, and in the diaries of daily activities that participants completed.
During the trialling exercises, each group (comprising 8-10 citizens) met on three occasions for around 90 minutes. Participants were asked keep a simple diary during the intervening periods between group meetings.
During the project's trialling phase, 157 people took part in 18 STAVE interventions. To enable comparative analyses of STAVE performance, we agreed that national trials would be encouraged to follow the same general approach to implementing group discussions. A degree of flexibility was necessary to allow individual teams to adapt their interventions to their specific national circumstances and needs. A consequence of this combined approach of comparability and flexibility is that the implementation of STAVE components varied slightly between countries, with partners developing specific instruments tailored to their national setting.
The organisation of the project work
PACHELBEL was organised according to a matrix pattern. In this way, partner organisations divided their time commitment between STAVE trialling work in their own countries, and project-wide work package responsibilities. Six work packages were identified: overall coordination and administration; the overall methodological integration and the in-house evaluation of the project; the investigation of the work of collaborating policy organisations; the development of stimulus material for the group process; the design of the group process; and the dissemination of project findings and outputs. Nine of the ten partners worked to implemented STAVE trials within their six countries. Seven of the principal researchers adopted lead roles of the six work package functions. The project scientific coordinator provided overall leadership, supported by her own organisation team, and by the project methodologist. Specific details of who did what is set out at section 4 (Potential Impacts).
3.3 Primary objectives: the STAVE trialling process and capturing STAVE design knowledge
Reiteration of the primary objectives
The project was concerned primarily with the design, development and operationalisation of the STAVE policymaking support tool. This process necessitated trialling of the tool prototype in the context of real-world policy issues. The project and the STAVE tool were both to be evaluated in a rigorous fashion by means in in-house processes. All these objectives are considered in this part of the report.
The other primary objectives: to produce guidance on how best to use STAVE across a range of policy environments, and to disseminate the findings of the project, are both considered in Part 4 of this report, as they are concerned with existing or potential impacts of the project.
The trialling process
At an early stage in the project, we took the decision that STAVE needed to be trialled in the context of live, real-world, policy contexts, rather than simulated ones. We took the view that only by engaging with naturally-occurring policy situations would we be able to gain suitable 'buy-in' from policymakers and citizens, and so gain intimate access to their everyday patterns of behaviour and practical reasoning.
As an exercise in action research, our trialling process was driven primarily by the policymakers' concerns and needs, rather than by matters of scholarly interest. Such research necessitates the active involvement of practitioners throughout the research activity. It also often requires the work to be done according to timescales that are sometimes unusually short in comparison with that of much academic work. As such, action research can present academics with something of a challenge. Indeed, previous such research within policy settings has often been in the context of collaborations between policymakers and practitioner communities. In this case, the research team's considerable experience in applied and policy-related research made this potential difficulty less of a problem than it might have been for some pure researchers. The PACHELBEL team's focus on working closely with policymakers, and using STAVE to deliver practical support for policymaking in real time, proved successful in by far most of the collaborations. In this way, the trialling of the STAVE tool was grounded within the everyday social worlds of policymaking and everyday citizen practices.
In each country, we worked with our collaborating policymakers to identify a live policy issue where they had an active interest in exploring or validating their knowledge about citizen's sustainability-related behaviours. These specific policy issues were explored with specially-recruited groups of citizens using the STAVE methodology.
Data generated during the STAVE interventions provided feedback to our policymaker partners. We sought to make this feedback both meaningful and constructive. We did so by providing two kinds of feedback: the first kind, 'speedy', which drew directly on the citizen participants' deliberations, using a variety of 'low tech' resources (e.g. oval maps, 'brainstorming' exercises, data from the resource allocation exercises etc.) generated by the group itself; and second, a more orthodox data analysis of the group conversations and activities, which was conducted by the research team after the group process was completed. This orthodox data analysis played an important role in validating and fine-tuning the 'speedy' mechanisms.
It is important to note that the policy organisations with which we worked presented a range of contrasting features, including: their experience of conducting or commissioning social research; their familiarity with various kinds of citizen engagement; and the extent to which they had previously considered in explicit ways the diversity of citizen behaviours and practices.
In assessing the wider value of STAVE as a policymaking support tool it is important to consider the contexts in which the application of STAVE took place. The policy organisations in Germany, Sweden, UK and in one of the Spanish interventions had most experience of having commissioned social research into public views about sustainability issues. The UK organisation had probably commissioned the most substantial research work on lay views about climate change. In contrast, the policy organisation from Romania was less familiar with social research, and also had less direct experience of how to interpret data generated by public engagement.
In France, two of the policy organisations had roles that routinely made use of the results from social science research. They regarded themselves as 'end users' of social science research, but did not see themselves as having an appropriate role in producing or commissioning it. The third French organisation had a more direct experience of citizen engagement. In Spain, the organisation involved in Agenda 21 was very familiar with participatory approaches, and most of the officials had backgrounds in environmental education. The second Spanish organisation recognized the potential value of social research to support policymaking, but had no practical experience in this regard. Thus, there were considerable variations in the policy organisations' backgrounds and levels of experience, within as well as between countries.
The policy area most thoroughly addressed by the STAVE trials was the topic of energy use. This was addressed in a number of ways: patterns of spending and saving electricity; heat and hot water; the use of smart meters and their relationship with electricity savings; and domestic insulation. Energy topics were the focus for STAVE work in France, Germany, Romania and Spain. Spain and Sweden both considered mobility issues. In the UK, the focus was on consumers' understanding of product lifetimes and its relationship with shopping behaviour.
Significantly, we found evidence that policymakers' involvement in the use of STAVE promoted an enhanced degree of reflection on their part on the ways in which they considered policy issues, and on their existing knowledge and assumptions about citizen behaviours. During the close and dynamic collaboration between the policymakers and members of the PACHELBEL team, a number of the policymakers indicated that they had come to reflect on their own work, as well as on evidence gaps regarding consumer behaviour that STAVE trials served to access. As a result of the collaborative engagement, some policymakers, for example in Spain, decided to change the focus of their choice for STAVE intervention after having realised that their real interest was on shopkeepers' behaviour rather than on business premises. The processes of dialogue involved in selecting a suitable policy issue for STAVE intervention appears to have prompted policymakers to evaluate more closely the evidence base on what policies work and why. In turn, these processes served to surface implicit, or ill-formed, assumptions about the scope for encouraging shifts in citizen behaviour.
Turning to the citizen discussion and diary process, the evidence is clear that STAVE is a method that is capable of generating a high degree of constructive engagement with groups of citizens. Such engagement is highly effective in eliciting patterns of socially-shared everyday behaviours, and authentic ways of talking about such behaviours. At first the citizen participants were cautious, but as they become more involved in the process they clearly displayed a sense of freedom to explain their informal doubts and contradictions; features of their everyday experience that we suspect would be difficult to capture using more conventional social research methods.
As noted above, we tested two modes of feedback; one which we termed 'speedy' and the other 'orthodox'. The first one presented policymakers with materials generated directly by participants during the group process. The second one took the form of classical report, generated through more structured analysis of data generated by the group process. In most cases, both stages of feedback took place during face-to-face meetings we had with policymakers. Our presentation of feedback provided policymakers with the opportunity to seek additional information and clarification as required.
Capturing STAVE design knowledge
Having carried out the STAVE trials, we then needed to analyse the considerable volume of data collected. Examination of this body of evidence allowed us to move towards a systematic understanding of the properties of the prototype STAVE tool in generating data corresponding to a range of concrete policy issues. In this way, we developed design knowledge that would allow us to use the STAVE tool in a targeted way to support policymaking for specific policy issues. The resulting design knowledge was assembled in the form of the STAVE Brochure and STAVE Manual, which are described in more detail in Part 4 below.
3.4 Secondary objectives: the social worlds of policymaking and everyday citizen behaviours
Reiteration of the secondary objectives
The project's secondary objectives sought to make use of the research access to naturalistic aspects of policymaking and citizen behaviours that were necessary in order to satisfy the primary, STAVE tool-related, objectives. In order to develop and trial the STAVE tool, we needed to gain fairly intimate access to the social worlds of policymaking and everyday consumer practices. Having successfully addressed our primary objectives, which were of a practical, policy-orientated, nature, we were able to re-examine our data with the more research-oriented secondary objectives in mind.
Our secondary objectives were concerned with investigating the nature of policymaking practices and processes concerned with sustainable consumption, with particular regard to the ways in which assumptions about human behaviour are incorporated into these practices. We were also concerned with the nature of practical reasoning by lay citizens about sustainable consumption, with particular regard to its relationship with their everyday consumer behaviours.
These secondary objectives were also concerned with the production of a range of scholarly publications, conference presentations and other outputs concerned with knowledge about these naturalistic aspects of policymaker and citizen behaviours. We will address the dissemination aspects of the secondary objectives in Part 4 below.
STAVE feedback and policy assumptions
The STAVE tool has shown its capacity to question assumptions that policymakers may have held about citizen behaviours. Most collaborating policymakers reacted to STAVE feedback with considerable interest, and sometimes surprise, as STAVE seemed to be able to 'bring alive' citizens' practices in a particularly lively and vivid way. The tool's capacity to show the real-world nature of the gap between discourses and practices of sustainability, and to be appreciated as such by policymakers, has been perhaps the most significant achievement of the project.
However, in a sense, the acronym STAVE ('Systematic Tool for Behavioural Assumption Validation and Exploration') that we adopted for the policymaking support tool we developed was something of a misnomer. The process of confronting policymakers with STAVE feedback has served to cast some doubt on whether clear assumptions about citizen behaviour were being incorporated into policy in the first place. Rather, it seems that a range of sometimes provisional and informal ideas - 'hunches' one might say - were being used in the process of policymaking. In itself, this is an important finding about the nature of policymaking practice.
In the cases where the policymaking organisations had commissioned research of their own, the ideas in use about citizen behaviours were more explicit and elaborated. Even in these circumstances, the ideas were often provisional and contested. Importantly, STAVE demonstrated its capacity to prompt policymaker reflections on what they knew and what they didn't know. The engendering of such processes of self-reflection seems to be an important property of STAVE as a policymaking support tool.
The practical accomplishment of policymaking
In working closely with a range of policymaking organisations, we were able to gain insights into the way that the everyday work of policymaking was done, in the real world of practice. These insights proved invaluable when it came to agreeing the nature of the STAVE interventions that we staged in order to support the specific policy issues with which our collaborators were engaged. It also helped us in seeking to present STAVE feedback in as helpful a way as possible. As we anticipated, in addition to helping us run the STAVE trials, these insights into the nature of policymaking work offered the possibility of significant research findings on the practical accomplishment of policymaking.
Although there exists a huge literature on the nature and work of policymaking organisations, very little of this material has been devoted to investigating naturalistic aspects of everyday policy work. As indicated earlier in this report, we recognised that capturing the dynamics of this work would necessitate a high degree of ethnographic immersion in the locations where the policy work was done. This degree of access was not possible in most of the organisations with which we worked. However, very significant volumes of such ethnographic data were collected in one location, with the potential to generate novel findings on the work of policymaking. The extent to which the intimate, but less intensive, data collection within the other collaborating organisations has the capacity to generate research findings is currently being explored during the preparation of publications.
Citizens' everyday sustainability-related practices
As discussed above, the development of the STAVE tool provided us with access to naturalistic aspects of the work of policymaking. It also provided access to naturalistic aspects of everyday citizen behaviours. The use of the STAVE tool-kit in different ways provided opportunities to investigate discourses of sustainability, the practices and accountability of citizen consumer practices, and the possibility of mismatch between formal commitments to sustainability and practical barriers to acting in sustainable ways. A significant volume of data was collected from the STAVE trials which allowed us to assemble comparative descriptions of citizen reasoning processes, as they related to: talking about and making sense of sustainability; living sustainably; and the gap between formal commitments or aspirations and real behaviours.
We found that in general, most participants in the STAVE groups displayed awareness that individual consumption practices concerning energy use, mobility and waste are strongly connected with sustainablility in the wider sense of environmental protection, climate change, nature conservation or responsibility for future generations. However, we found a great deal of evidence for a mismatch between framing consumption practices in terms of wider environmental issues or in terms of everyday domestic practices.
This mismatch constitutes a crucial methodological issue when seeking to investigate practical assumptions about citizen behaviours. Importantly, it provides strong empirical evidence to support our fundamental assumption about the gap between practical reasoning appropriate for policymaking for sustainability and practical reasoning for the conduct of everyday domestic life. These two spheres of human experience do indeed appear to constitute different forms of life. These dimensions of citizen sustainability-related experience will be explored in publications currently in preparation.
3.5 Evaluating the project and the STAVE tool
The in-house evaluation process
Throughout the course of the project, an in-house programme of evaluation was carried out. This activity began collecting data on the first day of the project and continued until the last day of the allotted lifetime of the work. The framework for evaluation drew upon technical aspects of evaluation theory and practice, and included measurement of performance in terms of process and outputs. The process measures drew upon recent evaluation research developments concerning the efficiency and effectiveness of the collection of data, and the flows of information between different stages of the work. The evaluation activity was designed to make extensive use of datasets that would be collected primarily as part of the central research and development activity, rather than specifically for evaluation. In this way, we set out to make the evaluation activity as unobtrusive and efficient as possible.
The roles of external interaction in evaluation: collaborative work, dissemination, international advisory committee and end-of-project workshops
In addition to the use of measures of internal effectiveness and efficiency, our interaction with the external world provided important opportunities to evaluate project performance. The successful development of the STAVE tool and its trialling in the context of live policy issues constitutes a powerful measure of the overall effectiveness of the project. The feedback received from collaborating policymakers was predominantly positive. In this way, we can point to firm evidence of the sound performance of the project.
A second external measure of effectiveness was provided by the interest shown by the scholarly community in our work. During the second year of the project, we proposed to the organisers of a major European conference that we present a collection of papers on our work in progress, and this was agreed and successfully executed. During the final year of the project, our work constituted dedicated symposia forming part of a major European conference, and a major international conference held in the United States. Feedback received from these occasions has been consistently positive. Our activity in disseminating the work and findings of the project at scholarly conference and other occasions will now form the basis for a series of peer-reviewed publications, which are currently in preparation. Members of our international advisory group have provided consistently supportive, but not uncritical, voices during the course of the work. Their capacity to force us to confront difficult issues, especially concerned with the overall framing and positioning of the work, has proved invaluable, and we are most grateful for their efforts.
Turning finally to the end-of-project workshops, we are delighted to be able to report, once again, on positive evaluations, delivered in such a way so as to fine-tune and enhance the quality of the project. In the main workshop, held in Brussels in September 2012, we examined the findings of the project with collaborating policy organisations and a number of officials from the European Commission. At the heart of the discussion were a series of joint presentations on the application of the STAVE tool to specific policy issues, presented by members of the project team with collaborating policymakers. Members of the international advisory committee joined in the discussions online. During the following month, we ran another workshop in Bucharest; this time focused on disseminating the project findings in Eastern Europe. Once again, we received positive feedback on what we had to offer.
The project's overall performance
Overall, the performance of project PACHELBEL has been highly satisfactory. It met all its objectives, including the design and operationalisation of the STAVE tool, and the trialling of that tool in the context of live policy settings. The feedback from our collaborating policy organisations has been strikingly positive. We feel that we are close to finalising STAVE as a potentially valuable contribution to the world of policymaking for sustainability.
A lively and authentic impression of the collaborating policy maker's enthusiasm for STAVE could already be gained at the 3rd Consortium Meeting (Stockholm) were some of them reported on STAVE:
-It did the job
-Addresses right topics
-Fosters a better understanding of needs of target groups
-Creates new ideas, approaches
-Delivers insights in values, norms, attitudes
-Good for ranking the value of different policy initiatives
-Delivers valuable and resilient ('robust') data for further internal consultation
-Flexible across different policy areas
-Very applicable in terms of costs and timescales
-Meaningful results in acceptable period of time
-Good at communicating questions to participants
PACHELBEL was essentially an action research project, in the sense that it was focused primarily on the achievement of practical tasks defined by the policy organisations with which we collaborated, rather than on objectives defined in terms of scholarly considerations. All such projects are pivotally dependent upon the establishment of satisfactory working relationships with collaborating bodies, which provide access to naturally-occurring work situations. Crucially, such research is also subject to the vagaries of the working environments in which the collaborating organisations operate. Given these considerations, it is especially important to stress the considerable success achieved by the project. The project team was able to implement STAVE trials across six different European countries, at different levels in public administration ranging from strategic to local, with a high degree of synchronisation, effectiveness, and compatibility.
We have a number of additional specific conclusions:
-STAVE is a user-friendly support tool that generates rich, grounded, data about everyday citizen behaviours in a relatively speedy way. It is a very flexible method, and can potentially be applied across a wide range of policy contexts.
-Despite being deceptively simple in design, STAVE requires deployment by facilitators with mature social research skills. Careful design work is needed to adapt the STAVE tool-kit to the specific research context. Mature social research capabilities are also needed in order to analyse the diverse data gathered, and to draw reliable conclusions.
-Different components of the STAVE toolkit seem to have different capacities to capture everyday features of sustainability-related citizen behaviours. The data generated by these different tools also varies in terms of its appeal to policymakers. It is perhaps the juxtaposition of diaries and face-to-face groups that provides one of the most distinctive elements of STAVE: its capacity to engender reflection on the relationship between 'in-principle' claims and grounded accounts of everyday practices.
-STAVE operates by brokering indigenous knowledges between the social worlds of policymaking and citizens' domestic and other everyday behaviours. In so doing, in creates intelligible communication between these distinct spheres of experience, action and rationality.
-In functioning as a policymaking support tool, STAVE also serves to promote critical reflection by policymakers of their existing knowledge, hunches and assumptions about what citizens do in their everyday lives. In this way, STAVE is able to enrich the policymaking process, by encouraging engagement by policymakers with authentic and grounded features of naturally-occurring citizen practices and rationalities.
-STAVE is not an orthodox citizen engagement tool. It is a policymaking support tool that produces forms of engagement with both citizen and policymakers. Policymaker involvement promotes self-refection about their own practices, and engenders sensitivity to citizens' cultural practices. As such, STAVE might be regarded as an effective means to generate capacity-building for both the policy community and civil society.
Potential Impact:
4. Existing and potential impacts
4.1 Contributions to addressing live policy issues: STAVE as a policy support tool
What worked well and what didn't work so well, and why
As already noted, the STAVE feedback that we generated for the policymakers was regarded by them as mostly useful and constructive. Most of the policy organisations involved in the project use social research in their activities in some way, usually to inform the design of policies to promote sustainability. In so doing, they have tended to set a clear boundary between their activities and the social research they commission or accessed in some other way. Significantly, the nature of STAVE interventions necessitates the active involvement of policymakers from the beginning of the process: from the formulation of the questions to be investigated; during the citizen group process, when they are invited to comment on preliminary feedback and perhaps refine their questions; to the final face-to-face engagement with the STAVE operators when the materials generated by the citizen groups are considered. For some of our policymaker collaborators, this degree of involvement in a research process seemed a little unusual. Yet, following their participation, most policymakers recognised the significant value of their involvement.
Most of the policymakers responded with interest, and sometimes surprise, when confronted with STAVE feedback. They mostly valued the opportunity to gain deeper insight into citizen behaviours, and the ways in which citizen rationalised those behaviours. In some cases the data confirmed their hunches and expectations. A key benefit, as recognised by policymakers in several countries, was the novel capacity of STAVE to shed some light on the problem we have already characterised as 'the gap between what citizens say and what they do'. A number of the policymakers were also pleasantly surprised by the capacity of STAVE to engender a willingness by citizen participants to talk openly about their domestic and other everyday practices. On occasions they were surprised by citizens' capacity to recognize, and reflect upon, the occurrence of inconsistencies between their environment-related practices and their accounts of those practices. Overall, a number of the policymakers were especially positive about how the immediate and unrefined feedback served to capture relevant evidence from citizens, and to make this available for the policymaking work in a relatively speedy, yet meaningful way.
The impact of STAVE
Turning now to how policymakers might use the STAVE findings, we have to acknowledge that the picture is more confused. It is not entirely clear how the STAVE interventions will, in the short term, influence policymaking and implementation concerning our target policy issues. Early comments from the collaborating policymakers have provided us with some encouragement about the potential future application and influence of STAVE. However, the eventual outcome seems likely to turn on the resolution of processes within the policy organisations. Of central importance, it seems, will be two questions that a number of the policymakers asked themselves: 'can this be used as an argument to be put forward to politicians? 'and 'does this help to determine where to concentrate our efforts? '
The perceived status of the STAVE findings in terms of how 'scientific' they might be regarded seems to be an important factor here. Generally, those more strategic policy organisations with which we worked found the early, 'immediate and unrefined', feedback less useful than the 'orthodox' feedback. Some regarded this material as very different in nature from the social research reports with which they were familiar. Importantly, the early feedback, in its somewhat raw and rich form, did not look like the fruits of standard social research. It is possible that such material was therefore not regarded as 'scientific', in some sense, and therefore could not be taken to politicians or senior decision-makers as evidence to justify action, especially if such action necessitated significant investment in terms of time and/or money. In contrast, the more locally-based policy organisations tended to find more immediate value in STAVE. Indeed, in places there was a real appetite to apply the findings, and to explore the applicability of STAVE in other policy areas.
It is clear from the evidence of the trials that there are limits to the extent to which STAVE is already able to provide what a number of policymakers would regard as a valuable support tool. However, it is important not to lose sight of the very significant positive messages that have emerged from the trials. The difficulties, such as they are, seem to be primarily concerned with finding ways of successfully embedding STAVE within the pre-existing patterns of practice and culture within policy organisations. We are currently seeking to address this obstacle by means of the preparations of materials that will allow prospective users to quickly understand the nature of the STAVE tool, and the ways in which it might provide specific policymaking support.
4.2 The STAVE Brochure and Manual
In line with our objective to produce guidance on how best to use the STAVE tool across a range of policy environments, we have used the STAVE design knowledge to prepare a STAVE Brochure and a STAVE Manual. The Brochure is a simple introduction to the STAVE tool, written with potential users in mind. The Manual is a more comprehensive resource prepared for potential users, which draws on case study material, and so supports realistic planning to use STAVE in specific policy contexts.
Both Brochure and Manual have been posted on the PACHELBEL website. Their initial format is as textual material, with links to supplementary visual and audio material. We plan to continue to enrich these resources, with our objective to produce multi-media web objects which include links to video materials.
4.3 Dissemination activities: project website, newsletter, conference presentations and publications
The team has been energetic in disseminating its work by means of a range of mechanisms. A PACHELBEL website has been maintained throughout the life of the project, containing up-to-date information about the project itself, and its events, findings and publications. The website continues to be maintained. Six issues of an illustrated electronic newsletter were produced and distributed widely. A poster and flyer were also produced as promotional materials. Members of the team contributed various talks, briefing and presentations to local, national and international events, including 18 papers to international conferences.
Some peer-reviewed papers have already started to appear, for example in the journal Periodistica, online in the series produced by the Spanish Communication Research Association, and in the working paper series published by the Swedish National defence College. Some are receiving minor revisions in the light of reviewer comments, for example a chapter from the book Knowledge Brokerage for a Sustainable Europe, and from the European Journal of Decision Processes. Others are currently under review or about to be submitted to journals like Energy Policy, Public Understanding of Science, Environmental Values, Environment and Behaviour, and Human Relations.
4.4 Next steps
Despite the success of project PACHELBEL, which we have described in this report, there exist a number of outstanding tasks which still require attention before STAVE can begin to play a significant role in policymaking for sustainability in Europe and beyond. At present, we are working on how best to put in place mechanisms that will allow these additional objectives to be satisfied.
Although a great deal of effort has already gone into disseminating the ideas, findings and practice of the project, establishing STAVE as well-known and well-regarded policymaking support tool is a potentially difficult and time-consuming process. STAVE has the potential to be useful across a wide range of policymaking contexts. Many policy organisations which could make use of STAVE may be difficult to contact, and they will almost certainly have a preference for what they regard as tried-and-tested ways of doing their work. They will also tend to only adopt new ways of doing things on recommendation from trusted contacts.
There is a potential circularity here. As the number of successful STAVE applications grows, and these can be documented and made available via the website, talks and publications, so the possibility of a more widespread use of STAVE will be become more likely. However, clearly developing this body of work necessitates the existence of wider access to policy organisations. Importantly, a number of policy organisations involved in PACHELBEL are playing a key role in this regard, as they are introducing the tool to other departments within their own organization.
Further developing the PACHELBEL website will be an important activity. As noted above, we plan to further enhance the STAVE Brochure and STAVE Good Practice Guide, by transforming them from primarily text-based resources into multi-media web objects, including audio and video material on our existing case studies. We are also investigating ways of making STAVE even more user-friendly, and to more effectively embed the tool within the work of policy organisations. Towards these ends, we are seeking new project work that will provide us with access to more naturally-occurring policy settings. Finally, we are exploring the means to establish an enduring network of STAVE users (STAVE Association), and a reservoir of case study-based STAVE application evidence. A first step in this regard is the creation of an STAVE Association, already being promoted by the project coordinators.
Conclusions
STAVE produces creative forms of engagement with both citizens and policymakers. Policymaker involvement in the STAVE process promotes self-refection about their own practices and their existing knowledge about citizen behaviours, and engenders sensitivity to citizens' indigenous practices. STAVE makes visible the nature and rationalities of citizens' environment-related practices by allowing citizens to discuss policy-related issues in their own everyday terms. Importantly, the tool can reveal the nature of practical barriers preventing the adoption of environmental-friendly consumer behaviours. The tool offers a means to allow policymakers to design and communicate their sustainability policies much more effectively.
Importantly:
-The use of STAVE is relatively inexpensive.
-STAVE is relatively speedy in use.
-STAVE is user-friendly for both citizens and policymakers.
-The feedback received from the collaborating policy organisations gaining support from STAVE has been strikingly positive.
Overall, the performance of project PACHELBEL has been highly satisfactory. It met all its objectives, including the design and operationalisation of the STAVE tool, and the trialling of that tool in the context of live policy settings.
PACHELBEL was essentially an action research project, in the sense that it was focused primarily on the achievement of practical tasks defined by the policy organisations with which we collaborated, rather than on objectives defined in terms of scholarly considerations. All such projects are pivotally dependent upon the establishment of satisfactory working relationships with collaborating bodies, which provide access to naturally-occurring work situations. Crucially, such research is also subject to the vagaries of the working environments in which the collaborating organisations operate. Given these considerations, it is especially important to stress the considerable success achieved by the project. The project team was able to implement STAVE trials across six different European countries, at different levels in public administration ranging from strategic to local, with a high degree of synchronisation, effectiveness, and compatibility. This is an impressive achievement.
List of Websites:
http://www.pachelbel.eu