Periodic Reporting for period 2 - weDRAW (Exploiting the best sensory modality for learning arithmetic and geometrical concepts based on multisensory interactive Information and Communication Technologies and serious games)
Berichtszeitraum: 2018-07-01 bis 2018-12-31
In the last years, multisensory technology for teaching has been increasingly used in the classroom. Whilst there has been a significant effort in developing technologies, such as the tangible, sensor, and mobile devices in the context of whole-body interaction and embodied interaction for learning, to date none of them sufficiently exploited multisensory integration for teaching and none of them has taken into consideration recent scientific findings in its development. weDRAW starts from the idea that each child has a preferential sensory channel to learn specific properties and that the visual signal is not always the most effective channel. Scientific studies show that specific sensory systems have specific roles for specific perceptual properties. Starting from such scientific ground the project proposes to create and evaluate a novel technology for deeper learning of arithmetic and geometry, based on multiple sensory modalities
Achievements were obtained at the scientific, pedagogical, and technological level.
At the scientific level, we applied psychophysical methodologies to understand the motoric capabilities of typical children at different stages of their development (6-7yo, 8-10yo). Moreover, we identified the most suitable sensory modalities to teach specific mathematical concepts. The technological solutions developed in weDRAW were validated at psychophysical and pedagogical level, by creating a link between these two approaches. We developed specific tests, and we performed psychophysical experiments with primary school children to evaluate their motor, arithmetic, and geometrical skills. The same skills were measured in visually impaired children. We validated the developed technologies with specific longitudinal trainings in classroom with sighted and visually impaired children. We also applied the weDRAW approach to children with dyslexia to identify whether multisensory trainings could be used to make screening and to train dyslexic children.
At the pedagogical level, we focused on the design and evaluation of the technological solutions proposed in the project. By working closely with primary school teachers, pedagogues identified key mathematical concepts for the target age group where multisensory engagement is particularly promising for a deeper understanding of such concepts. Teachers were involved in a Participatory Design process to iteratively inform the development of the technology and associated pedagogical activities. Moreover, we determined how and the extent to which the envisaged pedagogical framework can be applied to both typically developing and dyslexic and visual impaired children, in order to overcome barriers and promote social inclusion of impaired children. After the delivery of the final prototypes, and as part of the pedagogical summative evaluation, a series of qualitative empirical studies, with mainstream and visually impaired children in school contexts, was undertaken. Studies also included pre and post-tests around the specific mathematical constructs.
At the technological level, we developed libraries of software modules for analysis of nonverbal motoric and affective behaviour of children and for real-time control of auditory, haptic, and visual feedback. We also worked to an integrated hardware and software platform supporting multiple inputs and output devices, scalable to different learning environments, and supporting design and development of serious games. Finally, three serious games and additional learning activities were developed in an iterative development cycle. The first serious game concerns arithmetic, the second one addresses geometry, and the third one is a proof-of-concept of social learning. Tools for early diagnosis of dyslexia, grounding on evidence that problems in rhythm perception can be an indicator for dyslexia, were also released.
At the scientific level, we applied psychophysical methodologies to understand the motoric capabilities of typical children at different stages of their development (6-7yo, 8-10yo). Moreover, we identified the most suitable sensory modalities to teach specific mathematical concepts. The technological solutions developed in weDRAW were validated at psychophysical and pedagogical level, by creating a link between these two approaches. We developed specific tests, and we performed psychophysical experiments with primary school children to evaluate their motor, arithmetic, and geometrical skills. The same skills were measured in visually impaired children. We validated the developed technologies with specific longitudinal trainings in classroom with sighted and visually impaired children. We also applied the weDRAW approach to children with dyslexia to identify whether multisensory trainings could be used to make screening and to train dyslexic children.
At the pedagogical level, we focused on the design and evaluation of the technological solutions proposed in the project. By working closely with primary school teachers, pedagogues identified key mathematical concepts for the target age group where multisensory engagement is particularly promising for a deeper understanding of such concepts. Teachers were involved in a Participatory Design process to iteratively inform the development of the technology and associated pedagogical activities. Moreover, we determined how and the extent to which the envisaged pedagogical framework can be applied to both typically developing and dyslexic and visual impaired children, in order to overcome barriers and promote social inclusion of impaired children. After the delivery of the final prototypes, and as part of the pedagogical summative evaluation, a series of qualitative empirical studies, with mainstream and visually impaired children in school contexts, was undertaken. Studies also included pre and post-tests around the specific mathematical constructs.
At the technological level, we developed libraries of software modules for analysis of nonverbal motoric and affective behaviour of children and for real-time control of auditory, haptic, and visual feedback. We also worked to an integrated hardware and software platform supporting multiple inputs and output devices, scalable to different learning environments, and supporting design and development of serious games. Finally, three serious games and additional learning activities were developed in an iterative development cycle. The first serious game concerns arithmetic, the second one addresses geometry, and the third one is a proof-of-concept of social learning. Tools for early diagnosis of dyslexia, grounding on evidence that problems in rhythm perception can be an indicator for dyslexia, were also released.
weDRAW contributed to progress beyond the state of the art along with several different directions:
- Novel concepts and approaches: research in weDRAW studied the best sensory modalities for understanding specific arithmetic and geometrical concepts. Results in this direction were obtained in terms of a series of psychophysical studies. These results, integrated with the pedagogical inputs, informed design and development of technologies.
- A novel ICT-supported paradigm to learn arithmetic and geometry: the serious games and the activities weDRAW developed link body movement with visual and auditory feedback, taking art (namely music and drawing) as a fundamental basis to design feedback.
- A novel unique language: a relevant aspect in weDRAW is that its approach and technology enable the application of the same learning paradigm to typically developed and impaired children. weDRAW organized workshops to identify the best approaches to teach geometry to children with visual disability, and the prototypical activities weDRAW conceived for teaching some geometric concepts were tested with visually impaired children.
- Novel ICT tools for early diagnosis: research suggests that dyslexic children have a lower precision on time perception. To enable fast screening of dyslexia, an app was developed in weDRAW for assessing precision on time perception from different sensory modalities.
- A novel embodied and enactive pedagogical approach: weDRAW technologies contribute to innovative pedagogical approaches, enabling teachers to design activities meeting specific needs of their students. weDRAW focused a lot on teachers and their role. Teachers were involved in the design process of technology.
- Novel algorithms for analysis of nonverbal affective motoric and social interaction: the project worked on algorithms for analysis of nonverbal affective motoric interaction in the learning process. Techniques were integrated into the weDRAW libraries and platform.
- Novel ICT products and services for technology-enhanced learning. weDRAW released a platform, three serious games, and other prototypes that represent novel ICT products and services for technology-enhanced learning.
The success of the project was assessed through studies involving a group of children exposed to the developed multisensory learning paradigm and technology and a control group. Assessment concerned both psychophysical and pedagogical aspects. Both typically developing and impaired children were involved in the evaluation. Dyslexic children were also tested and trained with the same technology. Results showed that the use of multisensory serious games improves mathematical learning in children at elementary school. For example, the Spaceshape game improves the understanding of fractions and dimension in 7 year old children.
Results are expected to have a direct impact on society, especially for enhancing the teaching and learning processes, for increasing educational possibilities for pupils with special (dis)abilities, and for increasing the capacity to retain pupils in the areas threatened by lack of specialized schools. weDRAW is also expected to induce a chain of events encouraging both institutions and citizens to adopt more widely specialized ICT methods and tools into sustainable educational development models of the local and regional education systems. weDRAW consortium agreed that two serious games will be exploited: the Cartesian Garden by the SME Ignition Factory; the second game the RobotAngle game will be distributed by DeAgostini through its web-portal.
- Novel concepts and approaches: research in weDRAW studied the best sensory modalities for understanding specific arithmetic and geometrical concepts. Results in this direction were obtained in terms of a series of psychophysical studies. These results, integrated with the pedagogical inputs, informed design and development of technologies.
- A novel ICT-supported paradigm to learn arithmetic and geometry: the serious games and the activities weDRAW developed link body movement with visual and auditory feedback, taking art (namely music and drawing) as a fundamental basis to design feedback.
- A novel unique language: a relevant aspect in weDRAW is that its approach and technology enable the application of the same learning paradigm to typically developed and impaired children. weDRAW organized workshops to identify the best approaches to teach geometry to children with visual disability, and the prototypical activities weDRAW conceived for teaching some geometric concepts were tested with visually impaired children.
- Novel ICT tools for early diagnosis: research suggests that dyslexic children have a lower precision on time perception. To enable fast screening of dyslexia, an app was developed in weDRAW for assessing precision on time perception from different sensory modalities.
- A novel embodied and enactive pedagogical approach: weDRAW technologies contribute to innovative pedagogical approaches, enabling teachers to design activities meeting specific needs of their students. weDRAW focused a lot on teachers and their role. Teachers were involved in the design process of technology.
- Novel algorithms for analysis of nonverbal affective motoric and social interaction: the project worked on algorithms for analysis of nonverbal affective motoric interaction in the learning process. Techniques were integrated into the weDRAW libraries and platform.
- Novel ICT products and services for technology-enhanced learning. weDRAW released a platform, three serious games, and other prototypes that represent novel ICT products and services for technology-enhanced learning.
The success of the project was assessed through studies involving a group of children exposed to the developed multisensory learning paradigm and technology and a control group. Assessment concerned both psychophysical and pedagogical aspects. Both typically developing and impaired children were involved in the evaluation. Dyslexic children were also tested and trained with the same technology. Results showed that the use of multisensory serious games improves mathematical learning in children at elementary school. For example, the Spaceshape game improves the understanding of fractions and dimension in 7 year old children.
Results are expected to have a direct impact on society, especially for enhancing the teaching and learning processes, for increasing educational possibilities for pupils with special (dis)abilities, and for increasing the capacity to retain pupils in the areas threatened by lack of specialized schools. weDRAW is also expected to induce a chain of events encouraging both institutions and citizens to adopt more widely specialized ICT methods and tools into sustainable educational development models of the local and regional education systems. weDRAW consortium agreed that two serious games will be exploited: the Cartesian Garden by the SME Ignition Factory; the second game the RobotAngle game will be distributed by DeAgostini through its web-portal.