Final Report Summary - FASMED (Improving progress for lower achievers through Formative Assessment in Science and Mathematics Education)
Executive Summary:
The full final publishable summary report (with images and graphics) can be found in the attached Pdf.
FaSMEd Executive Summary
FaSMEd was a collaborative development project, which adapted the principles of design research (Swan, 2014) in its methodology. A consortium of international partners researched the role of technologically enhanced Formative Assessment (FA) methods with the view to developing a toolkit that would inform teachers of emergent FA pedagogies in mathematics and science education.
Through intervention cases in science and/or mathematics in each of the partner countries, innovative technology was introduced to create a digital environment which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations. The FaSMEd project explored the potential to amplify the quality of the evidence about student achievement both in real-time and outside the classroom for access by both students and teachers.
The FaSMEd project found that the introduction of innovative technology to create a digital environment (between students, peers and teachers) can assist teachers in making more timely formative interpretations. We recommend the use of such technologies within classrooms to further enhance FA practices.
Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers. We recommend that technologies are utilised within classrooms to facilitate making learning more visible to all ‘in the moment’.
Our FaSMEd case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. We would therefore recommend that when embarking on new technological innovations, the usability of tools is considered.
FaSMEd found that where existing infrastructures supported the use of technology, schools were able to make considerable progress in their use of technology to support FA practices. We would recommend investment in the networking and wireless systems, together with technical support in schools. FaSMEd believes this is a priority and a pre-requisite for the implementation of this technology on a larger scale.
Where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning. FaSMEd would therefore recommend that schools (wherever possible) facilitate time and space for teachers to plan, and reflect on their practice. A commitment to this from school leaders is crucial.
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6). The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
Project Context and Objectives:
The Rocard report (2007) identified widespread concern across the EU about the economic consequences and social impact of underachievement in mathematics and science education and recommended the adoption of an inquiry based pedagogy. As a consequence, a range of research projects were commissioned by the EC, for example: SAILS – Strategies for Assessment of Inquiry Learning in Science ; MASCIL – Mathematics and Science for Life ; PRIMAS – Promoting Inquiry in Mathematics and Science Education across Europe , and ASSIST-ME - Assess Inquiry in Science, Technology and Mathematics Education. FaSMEd – Formative Assessment in Science and Mathematics Education was the final project commissioned in the FP7 programme, with a specific remit to explore the application of technology to facilitate FA in the classroom.
FaSMEd was a collaborative development project, which has adapted the principles of design research (Swan, 2014; Burkhardt and Schoenfeld, 2003) into its methodology. This is a formative approach in which a product or process (or ‘tool’) is envisaged, designed, developed and refined through cycles of enactment, observation, analysis and redesign (Gravemeijer and Cobb, 2006), with trials in ‘real’ situations (Collins et al., 2004) and systematic feedback from end-users. Educational theory is used to inform the design and refinement of the tools, and is itself refined during the research process. Its goals are to create innovative tools for others to use, to describe and explain how these tools function, account for the range of implementations that occur and develop principles and theories that may guide future designs. Ultimately, the goal is transformative; we seek to create new teaching and learning possibilities and study their impact on end-users.
A key element of teaching using assessment and intervention relates to the quality of the information generated by the various feedback loops that exist in the classroom setting and the involvement of the students within this process. Through intervention cases in science and/or mathematics in each of the partner countries, innovative technology was introduced to create a digital environment which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations. The FaSMEd project explored the potential to amplify the quality of the evidence about student achievement both in real-time and outside the classroom for access by both students and teachers.
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6) . The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
The Professional Development (PD) package produced by FaSMEd reflects the range of ways in which partners have worked with teachers in their countries and offers examples for teachers and teacher educators to use. These include a set of six PD modules designed to help teachers use FA and technology more effectively in their classrooms. The resources also include a theoretical section on principles for effective professional development and a practical section on ways in which professional development can be organised. This section is meant to be used by people who are organising professional development for teachers of mathematics and science but can also be used by teachers either individually or working with peers.
Project Results:
Because of the nature of our S&T findings, it is necessary to include numerous pictorial images and so we include the entire description as an attached Pdf document.
Potential Impact:
Introduction
FaSMEd was a collaborative development project, which adapted the principles of design research (Swan, 2014) in its methodology. A consortium of international partners researched the role of technologically enhanced Formative Assessment (FA) methods with the view to developing a toolkit that would inform teachers of emergent FA pedagogies in mathematics and science education. By introducing innovative technology, we created environments which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations.
The FaSMEd Project
Formative Assessment (FA) strategies may be understood as follows:
‘Practice in a classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited’ (Black & Wiliam, 2009, p.9).
As such, they have been evidenced to impact on student learning (Black & Wiliam, 1998; Wiliam & Leahy, 2015). Hence there is a strong argument (and this is the rationale for the approach of FaSMEd) that embedding formative assessment strategies in teachers’ classroom practice will have a significant impact on students’ achievement. The professional development of teachers focused on FA is, therefore, a crucially important factor in the raising of achievement for all students and gives a clear direction of educational policy for leaders at institution, regional and national level.
The TIMSS seven-nation comparative study shows that high achieving countries (Hiebert et al., 2003) adopt approaches which preserve the complexity of concepts and methods, rather than simplifying them. FaSMEd partners were thus encouraged to create and adapt research-informed FA activities. Since these approaches increase the cognitive load for students we recognised that it was important that the learning environment was engineered to support students and FaSMEd included technology as part of the design of the environment to provide such support. Hence, FaSMEd partners were encouraged to develop resources, processes and technological tools which would allow all students to engage with complex concepts and methods successfully and to improve motivation. The FaSMEd project case studies provide examples of where this approach has worked successfully with lower achieving students.
Challenges of innovation
FaSMEd required many of the teachers to adapt to some fundamental innovations. The adoption of any innovation in education is challenging. Many of the FaSMEd resources required a pedagogy of engagement, which emphasized reflection over productivity and maintained a level of complexity, rather than the simplifying approach frequently adopted for lower achieving students. Adapting to a pedagogy of contingency in order to respond to the minute-by-minute feedback from their students and the integration of technological tools in their practice was demanding.
Recruiting schools that were willing to take risks in the process of innovation was not always straightforward. Those schools that did participate tended to have an established relationship of trust with the partners and therefore had the confidence to try something new. Such confidence and trust on the part of both teachers and the school leaders is important in implementing change.
In addition, change on this scale is likely to necessitate a significant amount of time. Wiliam (2016) warns that it is common to find that a significant impact on standardized test scores might only materialize after two to three years of implementation of a particular innovation. Policy makers, therefore, must be prepared to plan for and sustain change over an extended time scale.
FaSMEd findings: What makes a difference?
Technology facilitating Formative Assessment
FaSMEd researchers reasoned that a key element of teaching using FA and intervention relates to the quality of the information generated by the various feedback loops that exist in the classroom setting and the involvement of the students within this process. The introduction of innovative technology to create a digital environment which enhances connectivity and feedback between students, peers and teachers can assist teachers in making more timely formative interpretations. This further has the potential to amplify the quality of the evidence about student achievement, both in real-time and outside the classroom, for access by both students and teachers.
Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In the FaSMEd Framework (Figure 1) this represents providing feedback that moves learners forward by means of the Sending and Displaying functionality of technology, as well as potentially Activating Students as Instructional Resources for one another and, as a result of the activation of these strategies, Activating students as owners of their own learning. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers.
In addressing the needs of lower achievers in particular, a number of interventions used technologies that could be more easily accessible and did not demand high levels of literacy. Using polls and/or pictorial representations were shown to be useful in some circumstances. Polls had the further advantage of Processing and Analysing data in real time.
Where technologies were able to Provide an Interactive Environment, students could access a variety of tools to scaffold their learning. This enabled lower achieving students to engage more fully in tasks and therefore Activating students as owners of their own learning . Further, at the FaSMEd Final Meeting (Deliverable D8.4) it was argued that FA practices provide a meaningful reason for using technology in the classroom. Fullan & Donnelly argue:
“Up to this point, technology has not impacted schools. We agree with Diana Laurillard (2012) that technological investments have not been directed at changing the system but only as a matter of acquisitions. Billions have been invested with little thought to altering the learning system. There are also potentially destructive uses of technology on learning; we must beware of distractions, easy entertainment and personalisation to the point of limiting our exposure to new ideas. We focus not simply on the technology itself but on its use.” (Fullan & Donnelly, 2013, p.10).
Our case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. The case studies recognised that a limiting issue for a number of teachers was the ergonomic environment which produced connection difficulties in the system, increasing the time taken for the feedback to arrive from and to the students and therefore forming a potential obstacle to the adoption of the technology. Investment in the networking and wireless systems (and technical support available) in schools would seem to be a priority and a pre-requisite for the implementation of this technology on a larger scale.
Investing in teacher learning
It has been strongly argued that in order to bring about real change within schools, investing in the building of the capacity of teachers, with teachers being the key agents of change (Fullan, 2010), should be the main emphasis of schools’ policy (IPPR, 2013).
The FaSMEd case studies demonstrate how professional learning was facilitated through a number of structures: courses, direct work with individual teachers and/or teacher learning groups. These are exemplified through the Professional Development package . Our work with teachers has highlighted that where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning . As argued in our position paper on Professional Learning of teachers , we note that Professional Learning Communities (PLC) (Wenger, 1998) emerge as one of the most promising structures for professional learning, particularly when these involve collaborative inquiry (e.g. OECD, 2013; Ermeling, 2010; Nelson et al., 2008). Professional Learning Communities (PLC) emerge as one of the most promising structures for professional learning. FaSMEd teachers expressed the positive value of creating these environments, which are not always readily available in schools across Europe and South Africa.
FaSMEd conclusions and policy guidelines
• The FaSMEd project found that the introduction of innovative technology to create a digital environment (between students, peers and teachers) can assist teachers in making more timely formative interpretations. We recommend the use of such technologies within classrooms to further enhance FA practices.
• Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers. We recommend that technologies are utilised within classrooms to facilitate making learning more visible to all ‘in the moment’.
• Our FaSMEd case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. We would therefore recommend that when embarking on new technological innovations, the usability of tools is considered.
• FaSMEd found that where existing infrastructures supported the use of technology, schools were able to make considerable progress in their use of technology to support FA practices. We would recommend investment in the networking and wireless systems, together with technical support in schools. FaSMEd believes this is a priority and a pre-requisite for the implementation of this technology on a larger scale.
• Where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning . FaSMEd would therefore recommend that schools (wherever possible) facilitate time and space for teachers to plan, and reflect on their practice. A commitment to this from school leaders is crucial.
Disseminating the outcomes of FaSMEd
The FaSMEd Toolkit
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6) . The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
Professional Development package
The Professional Development (PD) package produced by FaSMEd reflects the range of ways in which partners have worked with teachers in their countries and offers examples for teachers and teacher educators to use. These include a set of six PD modules designed to help teachers use FA and technology more effectively in their classrooms. The resources also include a theoretical section on principles for effective professional development and a practical section on ways in which professional development can be organised. This section is meant to be used by people who are organising professional development for teachers of mathematics and science but can also be used by teachers either individually or working with peers.
The FaSMEd position paper on Professional Learning of teachers warned that Professional Development (PD) is perceived and experienced differently across countries. Partners were aware, therefore, that it was important not to assume too much about expectations and norms in other countries. However, the position paper then goes on to conclude that there is a high degree of convergence in descriptions of successful professional learning and the partners generally agreed. Typically, these include securing interest and engagement from the teachers, providing a theoretical framework for understanding of the innovation/strategy/programme and offering some practical tools to apply to classroom practice (Timperley et al., 2008).
The position paper also notes that Professional Learning Communities (PLC) (Wenger, 1998) emerge as one of the most promising structures for professional learning, particularly when these involve collaborative inquiry (e.g. OECD, 2013; Ermeling, 2010; Nelson et al., 2008). This is because the conditions for effective professional learning, fundamentally require teachers to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning (Joubert & Sutherland, 2008). Partners were thus encouraged to engage with groups of teachers who were willing to collaborate as active participants in the design process of the resources for the toolkit and to support PLC’s where possible.
In FaSMEd all partners used an active involvement of the teachers in the design-based research process as professional development. Teachers were involved through cluster meetings and school visits throughout the intervention phase of the project (2014/2015). These meetings included dialogues with the FaSMEd researchers, sharing of practice with other teachers as well as participating in the ‘design-do-review cycles’ of classroom materials. However, the organisation of this approach was very different for each FaSMEd partner but essentially fell into three main types: courses; learning groups and individual teachers .
Dissemination remained an ongoing objective throughout the project and we have worked effectively towards this, evidenced by our comprehensive dissemination activity (https://research.ncl.ac.uk/fasmed/disseminationactivity/ ). This included the production of outputs for academic, policy, practitioner, and student audiences; conference presentations and workshops, media reports and digital dissemination activities. We can report the following achievements and activities:
• Maintaining and updating the project website (see: https://research.ncl.ac.uk/fasmed/ ) (D7.1)
• Coordinating the production of a three monthly project newsletter (D7.2) (see https://research.ncl.ac.uk/fasmed/newsletters/ )
• Collating the many dissemination activities undertaken throughout the project. All are publicly available to download from:
https://research.ncl.ac.uk/fasmed/disseminationactivity/
• Partners have taken responsibility for delivering dissemination activities most applicable to their own countries and contexts, these include Stakeholder meetings (D7.7) held annually in each partner country.
• A coordinated strategy has been developed to communicate with European wide and international audiences (for example joint presentations at the European Educational Research Conference (ECER 2015), Commission internationale pour l’étude et l’amélioration de l’enseignement des mathématiques (CIEAEM) conference and a workshop on Assessment in Science and Mathematics at secondary level, Dublin). In order to facilitate these activities, briefing documents (D7.3) posters and booklets (D7.5) were produced by the end of month 12 of the project.
• A strategic effort has been made to identify other EU projects in the Science in Society theme and to communicate with them regarding possible areas for collaboration. The FaSMEd project is registered on the Scientix website and at present relationships have been formed with ASSIST-me, Sis-Catalyst, INSTEM, S-TEAM and SAILS, where we have discussed joint interests, shared appropriate materials and presented together at conferences (for example FaSMEd, SAILS and ASSIST-me presented together at the 2nd Scientix Conference, Brussels, as well as sharing an exhibition booth and distributing briefing papers; ASSIST-ME, SAILS and INSTEM presented at a workshop on assessment in science and mathematics at secondary level at Dublin City University, February 2015; and FaSMEd and SAILS presented a joint symposium at the European Science Education Research Association (ESERA) conference in September 2015).
• Dissemination and communication about the developing FaSMEd project was a recurring agenda item at UNEW project meetings, partner meetings, Strategic Advisory Committee meetings and with our External Adviser, Anne Liddon, who was recruited specifically to advise on dissemination. Anne Liddon attended our Inception meeting in this capacity and subsequent meetings have informed our continuing dissemination strategy. In particular, we sought advice on the recruitment and planning of our FaSMEd film documentary.
• D7.4 The FaSMEd Film. Following a tender procedure, MLM Learning Design were commissioned by UNEW, the company have considerable experience in working with schools, both in the UK and internationally, and in producing film focused on teaching and learning. UNEW worked collaboratively with MLM Learning Design in the design and production of the film, with filming of FaSMEd lessons taken place in the UK, Ireland, Italy and France. Additionally, film recordings produced by partners has also been incorporated into the produced FaSMEd film so that as many partners as possible are represented.
• An additional resource we had not originally planned in our DoW was achieved during this second reporting period: The FaSMEd Comic (see https://research.ncl.ac.uk/fasmed/disseminationactivity/FaSMEd%20Comic.pdf ). Comics Boss’ Lydia Wysocki from Applied Comics Etc (http://www.appliedcomicsetc.com/ ) worked with FaSMEd students to run a lunchtime comics club at one UK School, helping students age 12-13 to make comics as a way to reflect on their new-style lessons. This was a unique opportunity to capture the genuine student perspective of the FaSMEd project.
• D7.6 International Conference. Our Mid-Term Reviewers in month 20 suggested we economised for the remainder of the project and that we should consider limiting or combining the numbers of meetings we held. We therefore proposed at our Phase 3 launch meeting (Deliverable 8.3) that we should merge our two forthcoming events as suggested: our International Conference (D7.6) and our Final Meeting (Deliverable D8.4) which is stated in our DoW to report by month 36. We agreed to merge these two events into a single two-day event (the joint International Conference and Final Meeting D8.4) which took take place in November 2016 (month 35) at the University of Maynooth, Ireland. The event drew together the consortium partners, members of the Strategic Advisory Committee, Evaluation team and key invited participants. This represented a significant international academic community of experience, expertise and practice in science and mathematics education with specific knowledge of digital technologies and Formative Assessment. The conference focused on disseminating the outcomes of the project and was more ‘outward’ facing. The final meeting facilitated discussions around raising achievement in mathematics and science education with a focus on implications for future research and policies and our final deliverables due at the end of the project:
References
Barzel, B., Selter, C. (2015). The DZLM design principles for in-service teacher education. JMD, Journal für Mathematik-Didaktik, 36(2), 259–284.
Black, P., & Wiliam, D. (1998). Inside the Black Box. London: King's College School of Education.
Black, P., & Wiliam, D., (2009). Developing the theory of formative assessment. Educational Assessment Evaluation and Accountability, 21(1), pp. 5-31.
Ermeling, B. A. (2010). Tracing the effects of teacher inquiry on classroom practice. Teaching and teacher education, 26(3), 377-388.
Fullan M (2010) All systems go: the change imperative for whole system reform, Thousand Oaks: Corwin.
Fullan, M. & Donnelly, K. (2013) Alive in the Swamp: Assessing Digital Innovations in Education. London: Nesta
Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., Jacobs, J., Stigler, J. (2003). Teaching Mathematics in Seven Countries: Results From the TIMSS 1999 Video Study. Washington, DC.: U.S. Department of Education National Center for Education Statistics.
IPPR (2013) Excellence and equity: Tackling educational disadvantage in England’s secondary schools http://www.ippr.org/publications/excellence-and-equity-tackling-educational-disadvantage-in-englands-secondary-schools Accessed 21/11/2016.
Laurillard, D. (2012) Teaching as a Design Science London: Routledge.
Nelson, T. H., Slavit, D., Perkins, M., & Hathorn, T. (2008). A culture of collaborative inquiry: Learning to develop and support professional learning communities. Teachers College Record, 110(6), 1269–1303.
OECD (2013) TALIS report: Fostering learning communities amongst teachers. Retrieved from www.oecd.org/talis
Swan, M, (2014). Design Research in Mathematics Education. In Lerman, S, (Ed.), Encyclopedia of Mathematics Education Dordrecht, Springer.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge: Cambridge University Press.
Wiliam, D. (2016) Leadership for Teacher Learning West Palm Beach, FL: Learning Sciences International.
Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for K-12 classrooms. West Palm Beach, FL: Learning Sciences International.
Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: what will it take to make it work? In C. A. Dwyer (Ed.), The future of assessment: shaping teaching and learning (pp. 53-82). Mahwah, NJ: Lawrence Erlbaum Associates.
List of Websites:
https://research.ncl.ac.uk/fasmed/ and http://www.fasmed.eu
The full final publishable summary report (with images and graphics) can be found in the attached Pdf.
FaSMEd Executive Summary
FaSMEd was a collaborative development project, which adapted the principles of design research (Swan, 2014) in its methodology. A consortium of international partners researched the role of technologically enhanced Formative Assessment (FA) methods with the view to developing a toolkit that would inform teachers of emergent FA pedagogies in mathematics and science education.
Through intervention cases in science and/or mathematics in each of the partner countries, innovative technology was introduced to create a digital environment which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations. The FaSMEd project explored the potential to amplify the quality of the evidence about student achievement both in real-time and outside the classroom for access by both students and teachers.
The FaSMEd project found that the introduction of innovative technology to create a digital environment (between students, peers and teachers) can assist teachers in making more timely formative interpretations. We recommend the use of such technologies within classrooms to further enhance FA practices.
Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers. We recommend that technologies are utilised within classrooms to facilitate making learning more visible to all ‘in the moment’.
Our FaSMEd case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. We would therefore recommend that when embarking on new technological innovations, the usability of tools is considered.
FaSMEd found that where existing infrastructures supported the use of technology, schools were able to make considerable progress in their use of technology to support FA practices. We would recommend investment in the networking and wireless systems, together with technical support in schools. FaSMEd believes this is a priority and a pre-requisite for the implementation of this technology on a larger scale.
Where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning. FaSMEd would therefore recommend that schools (wherever possible) facilitate time and space for teachers to plan, and reflect on their practice. A commitment to this from school leaders is crucial.
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6). The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
Project Context and Objectives:
The Rocard report (2007) identified widespread concern across the EU about the economic consequences and social impact of underachievement in mathematics and science education and recommended the adoption of an inquiry based pedagogy. As a consequence, a range of research projects were commissioned by the EC, for example: SAILS – Strategies for Assessment of Inquiry Learning in Science ; MASCIL – Mathematics and Science for Life ; PRIMAS – Promoting Inquiry in Mathematics and Science Education across Europe , and ASSIST-ME - Assess Inquiry in Science, Technology and Mathematics Education. FaSMEd – Formative Assessment in Science and Mathematics Education was the final project commissioned in the FP7 programme, with a specific remit to explore the application of technology to facilitate FA in the classroom.
FaSMEd was a collaborative development project, which has adapted the principles of design research (Swan, 2014; Burkhardt and Schoenfeld, 2003) into its methodology. This is a formative approach in which a product or process (or ‘tool’) is envisaged, designed, developed and refined through cycles of enactment, observation, analysis and redesign (Gravemeijer and Cobb, 2006), with trials in ‘real’ situations (Collins et al., 2004) and systematic feedback from end-users. Educational theory is used to inform the design and refinement of the tools, and is itself refined during the research process. Its goals are to create innovative tools for others to use, to describe and explain how these tools function, account for the range of implementations that occur and develop principles and theories that may guide future designs. Ultimately, the goal is transformative; we seek to create new teaching and learning possibilities and study their impact on end-users.
A key element of teaching using assessment and intervention relates to the quality of the information generated by the various feedback loops that exist in the classroom setting and the involvement of the students within this process. Through intervention cases in science and/or mathematics in each of the partner countries, innovative technology was introduced to create a digital environment which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations. The FaSMEd project explored the potential to amplify the quality of the evidence about student achievement both in real-time and outside the classroom for access by both students and teachers.
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6) . The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
The Professional Development (PD) package produced by FaSMEd reflects the range of ways in which partners have worked with teachers in their countries and offers examples for teachers and teacher educators to use. These include a set of six PD modules designed to help teachers use FA and technology more effectively in their classrooms. The resources also include a theoretical section on principles for effective professional development and a practical section on ways in which professional development can be organised. This section is meant to be used by people who are organising professional development for teachers of mathematics and science but can also be used by teachers either individually or working with peers.
Project Results:
Because of the nature of our S&T findings, it is necessary to include numerous pictorial images and so we include the entire description as an attached Pdf document.
Potential Impact:
Introduction
FaSMEd was a collaborative development project, which adapted the principles of design research (Swan, 2014) in its methodology. A consortium of international partners researched the role of technologically enhanced Formative Assessment (FA) methods with the view to developing a toolkit that would inform teachers of emergent FA pedagogies in mathematics and science education. By introducing innovative technology, we created environments which enhanced connectivity and feedback to assist teachers in making more timely formative interpretations.
The FaSMEd Project
Formative Assessment (FA) strategies may be understood as follows:
‘Practice in a classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited’ (Black & Wiliam, 2009, p.9).
As such, they have been evidenced to impact on student learning (Black & Wiliam, 1998; Wiliam & Leahy, 2015). Hence there is a strong argument (and this is the rationale for the approach of FaSMEd) that embedding formative assessment strategies in teachers’ classroom practice will have a significant impact on students’ achievement. The professional development of teachers focused on FA is, therefore, a crucially important factor in the raising of achievement for all students and gives a clear direction of educational policy for leaders at institution, regional and national level.
The TIMSS seven-nation comparative study shows that high achieving countries (Hiebert et al., 2003) adopt approaches which preserve the complexity of concepts and methods, rather than simplifying them. FaSMEd partners were thus encouraged to create and adapt research-informed FA activities. Since these approaches increase the cognitive load for students we recognised that it was important that the learning environment was engineered to support students and FaSMEd included technology as part of the design of the environment to provide such support. Hence, FaSMEd partners were encouraged to develop resources, processes and technological tools which would allow all students to engage with complex concepts and methods successfully and to improve motivation. The FaSMEd project case studies provide examples of where this approach has worked successfully with lower achieving students.
Challenges of innovation
FaSMEd required many of the teachers to adapt to some fundamental innovations. The adoption of any innovation in education is challenging. Many of the FaSMEd resources required a pedagogy of engagement, which emphasized reflection over productivity and maintained a level of complexity, rather than the simplifying approach frequently adopted for lower achieving students. Adapting to a pedagogy of contingency in order to respond to the minute-by-minute feedback from their students and the integration of technological tools in their practice was demanding.
Recruiting schools that were willing to take risks in the process of innovation was not always straightforward. Those schools that did participate tended to have an established relationship of trust with the partners and therefore had the confidence to try something new. Such confidence and trust on the part of both teachers and the school leaders is important in implementing change.
In addition, change on this scale is likely to necessitate a significant amount of time. Wiliam (2016) warns that it is common to find that a significant impact on standardized test scores might only materialize after two to three years of implementation of a particular innovation. Policy makers, therefore, must be prepared to plan for and sustain change over an extended time scale.
FaSMEd findings: What makes a difference?
Technology facilitating Formative Assessment
FaSMEd researchers reasoned that a key element of teaching using FA and intervention relates to the quality of the information generated by the various feedback loops that exist in the classroom setting and the involvement of the students within this process. The introduction of innovative technology to create a digital environment which enhances connectivity and feedback between students, peers and teachers can assist teachers in making more timely formative interpretations. This further has the potential to amplify the quality of the evidence about student achievement, both in real-time and outside the classroom, for access by both students and teachers.
Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In the FaSMEd Framework (Figure 1) this represents providing feedback that moves learners forward by means of the Sending and Displaying functionality of technology, as well as potentially Activating Students as Instructional Resources for one another and, as a result of the activation of these strategies, Activating students as owners of their own learning. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers.
In addressing the needs of lower achievers in particular, a number of interventions used technologies that could be more easily accessible and did not demand high levels of literacy. Using polls and/or pictorial representations were shown to be useful in some circumstances. Polls had the further advantage of Processing and Analysing data in real time.
Where technologies were able to Provide an Interactive Environment, students could access a variety of tools to scaffold their learning. This enabled lower achieving students to engage more fully in tasks and therefore Activating students as owners of their own learning . Further, at the FaSMEd Final Meeting (Deliverable D8.4) it was argued that FA practices provide a meaningful reason for using technology in the classroom. Fullan & Donnelly argue:
“Up to this point, technology has not impacted schools. We agree with Diana Laurillard (2012) that technological investments have not been directed at changing the system but only as a matter of acquisitions. Billions have been invested with little thought to altering the learning system. There are also potentially destructive uses of technology on learning; we must beware of distractions, easy entertainment and personalisation to the point of limiting our exposure to new ideas. We focus not simply on the technology itself but on its use.” (Fullan & Donnelly, 2013, p.10).
Our case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. The case studies recognised that a limiting issue for a number of teachers was the ergonomic environment which produced connection difficulties in the system, increasing the time taken for the feedback to arrive from and to the students and therefore forming a potential obstacle to the adoption of the technology. Investment in the networking and wireless systems (and technical support available) in schools would seem to be a priority and a pre-requisite for the implementation of this technology on a larger scale.
Investing in teacher learning
It has been strongly argued that in order to bring about real change within schools, investing in the building of the capacity of teachers, with teachers being the key agents of change (Fullan, 2010), should be the main emphasis of schools’ policy (IPPR, 2013).
The FaSMEd case studies demonstrate how professional learning was facilitated through a number of structures: courses, direct work with individual teachers and/or teacher learning groups. These are exemplified through the Professional Development package . Our work with teachers has highlighted that where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning . As argued in our position paper on Professional Learning of teachers , we note that Professional Learning Communities (PLC) (Wenger, 1998) emerge as one of the most promising structures for professional learning, particularly when these involve collaborative inquiry (e.g. OECD, 2013; Ermeling, 2010; Nelson et al., 2008). Professional Learning Communities (PLC) emerge as one of the most promising structures for professional learning. FaSMEd teachers expressed the positive value of creating these environments, which are not always readily available in schools across Europe and South Africa.
FaSMEd conclusions and policy guidelines
• The FaSMEd project found that the introduction of innovative technology to create a digital environment (between students, peers and teachers) can assist teachers in making more timely formative interpretations. We recommend the use of such technologies within classrooms to further enhance FA practices.
• Through the case studies there is evidence of teachers using technologies to gain information about their students’ thinking, as well as to facilitate opportunities for students to learn from their peers. In interviews, students identified these practices as particularly beneficial in making their learning visible to the teacher, themselves and their peers. We recommend that technologies are utilised within classrooms to facilitate making learning more visible to all ‘in the moment’.
• Our FaSMEd case studies show that most teachers opted for technology tools which were accessible and/or easy to learn how to use and apply in their classrooms. We would therefore recommend that when embarking on new technological innovations, the usability of tools is considered.
• FaSMEd found that where existing infrastructures supported the use of technology, schools were able to make considerable progress in their use of technology to support FA practices. We would recommend investment in the networking and wireless systems, together with technical support in schools. FaSMEd believes this is a priority and a pre-requisite for the implementation of this technology on a larger scale.
• Where teachers were able to work as professional learning communities, conditions were effective in enabling them to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning . FaSMEd would therefore recommend that schools (wherever possible) facilitate time and space for teachers to plan, and reflect on their practice. A commitment to this from school leaders is crucial.
Disseminating the outcomes of FaSMEd
The FaSMEd Toolkit
The main objectives for the FaSMEd project were to produce (through design research) a Toolkit for teachers and teacher educators (Deliverable D3.3) and a Professional Development (PD) resource (Deliverable D3.6) . The expression ‘toolkit’ refers to a set of curriculum materials and methods for pedagogical intervention. These were designed to support the development of practice and are disseminated through a website produced by the partners, and can be accessed at: http://fasmed.eu.
Professional Development package
The Professional Development (PD) package produced by FaSMEd reflects the range of ways in which partners have worked with teachers in their countries and offers examples for teachers and teacher educators to use. These include a set of six PD modules designed to help teachers use FA and technology more effectively in their classrooms. The resources also include a theoretical section on principles for effective professional development and a practical section on ways in which professional development can be organised. This section is meant to be used by people who are organising professional development for teachers of mathematics and science but can also be used by teachers either individually or working with peers.
The FaSMEd position paper on Professional Learning of teachers warned that Professional Development (PD) is perceived and experienced differently across countries. Partners were aware, therefore, that it was important not to assume too much about expectations and norms in other countries. However, the position paper then goes on to conclude that there is a high degree of convergence in descriptions of successful professional learning and the partners generally agreed. Typically, these include securing interest and engagement from the teachers, providing a theoretical framework for understanding of the innovation/strategy/programme and offering some practical tools to apply to classroom practice (Timperley et al., 2008).
The position paper also notes that Professional Learning Communities (PLC) (Wenger, 1998) emerge as one of the most promising structures for professional learning, particularly when these involve collaborative inquiry (e.g. OECD, 2013; Ermeling, 2010; Nelson et al., 2008). This is because the conditions for effective professional learning, fundamentally require teachers to feel safe to experiment, examine the impact of their innovations, to talk openly and to establish principles about effective student learning (Joubert & Sutherland, 2008). Partners were thus encouraged to engage with groups of teachers who were willing to collaborate as active participants in the design process of the resources for the toolkit and to support PLC’s where possible.
In FaSMEd all partners used an active involvement of the teachers in the design-based research process as professional development. Teachers were involved through cluster meetings and school visits throughout the intervention phase of the project (2014/2015). These meetings included dialogues with the FaSMEd researchers, sharing of practice with other teachers as well as participating in the ‘design-do-review cycles’ of classroom materials. However, the organisation of this approach was very different for each FaSMEd partner but essentially fell into three main types: courses; learning groups and individual teachers .
Dissemination remained an ongoing objective throughout the project and we have worked effectively towards this, evidenced by our comprehensive dissemination activity (https://research.ncl.ac.uk/fasmed/disseminationactivity/ ). This included the production of outputs for academic, policy, practitioner, and student audiences; conference presentations and workshops, media reports and digital dissemination activities. We can report the following achievements and activities:
• Maintaining and updating the project website (see: https://research.ncl.ac.uk/fasmed/ ) (D7.1)
• Coordinating the production of a three monthly project newsletter (D7.2) (see https://research.ncl.ac.uk/fasmed/newsletters/ )
• Collating the many dissemination activities undertaken throughout the project. All are publicly available to download from:
https://research.ncl.ac.uk/fasmed/disseminationactivity/
• Partners have taken responsibility for delivering dissemination activities most applicable to their own countries and contexts, these include Stakeholder meetings (D7.7) held annually in each partner country.
• A coordinated strategy has been developed to communicate with European wide and international audiences (for example joint presentations at the European Educational Research Conference (ECER 2015), Commission internationale pour l’étude et l’amélioration de l’enseignement des mathématiques (CIEAEM) conference and a workshop on Assessment in Science and Mathematics at secondary level, Dublin). In order to facilitate these activities, briefing documents (D7.3) posters and booklets (D7.5) were produced by the end of month 12 of the project.
• A strategic effort has been made to identify other EU projects in the Science in Society theme and to communicate with them regarding possible areas for collaboration. The FaSMEd project is registered on the Scientix website and at present relationships have been formed with ASSIST-me, Sis-Catalyst, INSTEM, S-TEAM and SAILS, where we have discussed joint interests, shared appropriate materials and presented together at conferences (for example FaSMEd, SAILS and ASSIST-me presented together at the 2nd Scientix Conference, Brussels, as well as sharing an exhibition booth and distributing briefing papers; ASSIST-ME, SAILS and INSTEM presented at a workshop on assessment in science and mathematics at secondary level at Dublin City University, February 2015; and FaSMEd and SAILS presented a joint symposium at the European Science Education Research Association (ESERA) conference in September 2015).
• Dissemination and communication about the developing FaSMEd project was a recurring agenda item at UNEW project meetings, partner meetings, Strategic Advisory Committee meetings and with our External Adviser, Anne Liddon, who was recruited specifically to advise on dissemination. Anne Liddon attended our Inception meeting in this capacity and subsequent meetings have informed our continuing dissemination strategy. In particular, we sought advice on the recruitment and planning of our FaSMEd film documentary.
• D7.4 The FaSMEd Film. Following a tender procedure, MLM Learning Design were commissioned by UNEW, the company have considerable experience in working with schools, both in the UK and internationally, and in producing film focused on teaching and learning. UNEW worked collaboratively with MLM Learning Design in the design and production of the film, with filming of FaSMEd lessons taken place in the UK, Ireland, Italy and France. Additionally, film recordings produced by partners has also been incorporated into the produced FaSMEd film so that as many partners as possible are represented.
• An additional resource we had not originally planned in our DoW was achieved during this second reporting period: The FaSMEd Comic (see https://research.ncl.ac.uk/fasmed/disseminationactivity/FaSMEd%20Comic.pdf ). Comics Boss’ Lydia Wysocki from Applied Comics Etc (http://www.appliedcomicsetc.com/ ) worked with FaSMEd students to run a lunchtime comics club at one UK School, helping students age 12-13 to make comics as a way to reflect on their new-style lessons. This was a unique opportunity to capture the genuine student perspective of the FaSMEd project.
• D7.6 International Conference. Our Mid-Term Reviewers in month 20 suggested we economised for the remainder of the project and that we should consider limiting or combining the numbers of meetings we held. We therefore proposed at our Phase 3 launch meeting (Deliverable 8.3) that we should merge our two forthcoming events as suggested: our International Conference (D7.6) and our Final Meeting (Deliverable D8.4) which is stated in our DoW to report by month 36. We agreed to merge these two events into a single two-day event (the joint International Conference and Final Meeting D8.4) which took take place in November 2016 (month 35) at the University of Maynooth, Ireland. The event drew together the consortium partners, members of the Strategic Advisory Committee, Evaluation team and key invited participants. This represented a significant international academic community of experience, expertise and practice in science and mathematics education with specific knowledge of digital technologies and Formative Assessment. The conference focused on disseminating the outcomes of the project and was more ‘outward’ facing. The final meeting facilitated discussions around raising achievement in mathematics and science education with a focus on implications for future research and policies and our final deliverables due at the end of the project:
References
Barzel, B., Selter, C. (2015). The DZLM design principles for in-service teacher education. JMD, Journal für Mathematik-Didaktik, 36(2), 259–284.
Black, P., & Wiliam, D. (1998). Inside the Black Box. London: King's College School of Education.
Black, P., & Wiliam, D., (2009). Developing the theory of formative assessment. Educational Assessment Evaluation and Accountability, 21(1), pp. 5-31.
Ermeling, B. A. (2010). Tracing the effects of teacher inquiry on classroom practice. Teaching and teacher education, 26(3), 377-388.
Fullan M (2010) All systems go: the change imperative for whole system reform, Thousand Oaks: Corwin.
Fullan, M. & Donnelly, K. (2013) Alive in the Swamp: Assessing Digital Innovations in Education. London: Nesta
Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., Jacobs, J., Stigler, J. (2003). Teaching Mathematics in Seven Countries: Results From the TIMSS 1999 Video Study. Washington, DC.: U.S. Department of Education National Center for Education Statistics.
IPPR (2013) Excellence and equity: Tackling educational disadvantage in England’s secondary schools http://www.ippr.org/publications/excellence-and-equity-tackling-educational-disadvantage-in-englands-secondary-schools Accessed 21/11/2016.
Laurillard, D. (2012) Teaching as a Design Science London: Routledge.
Nelson, T. H., Slavit, D., Perkins, M., & Hathorn, T. (2008). A culture of collaborative inquiry: Learning to develop and support professional learning communities. Teachers College Record, 110(6), 1269–1303.
OECD (2013) TALIS report: Fostering learning communities amongst teachers. Retrieved from www.oecd.org/talis
Swan, M, (2014). Design Research in Mathematics Education. In Lerman, S, (Ed.), Encyclopedia of Mathematics Education Dordrecht, Springer.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge: Cambridge University Press.
Wiliam, D. (2016) Leadership for Teacher Learning West Palm Beach, FL: Learning Sciences International.
Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for K-12 classrooms. West Palm Beach, FL: Learning Sciences International.
Wiliam, D., & Thompson, M. (2007). Integrating assessment with instruction: what will it take to make it work? In C. A. Dwyer (Ed.), The future of assessment: shaping teaching and learning (pp. 53-82). Mahwah, NJ: Lawrence Erlbaum Associates.
List of Websites:
https://research.ncl.ac.uk/fasmed/ and http://www.fasmed.eu
