Periodic Reporting for period 3 - EQIPD (European Quality In Preclinical Data)
Reporting period: 2020-01-01 to 2021-09-30
The pharmaceutical industry, as well as the advance of biomedical science, depend on robust data and scientific rigor for efficient decision making, patent strength and reduced time-to-market, which in turn determines knowledge gain and availability of new treatments to patients. Increasing awareness of shortcomings in the robustness, rigor, and validity of research data reduce confidence in decisions on further preclinical or clinical testing and for the predictability of preclinical models.
We set out to propose simple, sustainable solutions to facilitate data quality without impacting innovation and freedom of research. We pooled resources from academia and industry to develop this action in Neuroscience and Safety, but with an ambition to have broader applicability.
We have used complementary approaches to [1] identify those factors associated with increased robustness, rigour and validity of in vivo research; [2] develop a quality framework for individual studies and for research groups; [3] pilot that framework; [4] refine the quality framework based on these pilot studies; and [5] put in place arrangements for the sustainability of the framework.
Conclusions of the Action: The project management WP1 guided the consortium in the timely completion of Milestones and Deliverables in fulfilment of the consortium’s contractual duties and effective dissemination and communication activities. In WP2, the evaluation of published and consortium data across three common paradigms (Irwin, EEG, OFT) was challenging due to limited reporting and study compatibility, but we highlighted specific areas for improvement, such as reporting of measures to reduce the risk of bias and specific aspects of experimental design within each paradigm. In WP3 we developed guidance for researchers working with animal experiments to increase rigour in design, conduct and analysis, based on a systematic review of existing guidelines and prospectively tested for feasibility by project partners. In WP4 we conducted multicentre studies and showed that standardisation and heterogenisation of protocols (open field), detailed definition of qualitative endpoints (Irwin), and centralized data analysis (EEG) could each reduce between lab variation. Drawing from this experience, WP5 developed a Quality System (QS) and associated tools to support scientists conducting non-regulated preclinical research. These have been released for public use and serve as a basis for building a sustainable post-funding future. WP6 have articulated governance elements for the EQIPD QS. We have gone beyond simply exploring sustainability options by creating a non-profit follow up organisation, the Guarantors of EQIPD, which will provide oversight to organisations wishing to provide EQIPD QS assessments. We developed a web-based learning environment in WP7 to learning opportunities tailored to early career researchers working in industry or academia with formal learning provided in an E-learning course and summer school. In WP8 we developed a new ontology for metadata of in vivo preclinical neuroscience experiments and have secured arrangements for data availability beyond the project. WP9 provided ethical insights to and oversight of our activities.
We set out to propose simple, sustainable solutions to facilitate data quality without impacting innovation and freedom of research. We pooled resources from academia and industry to develop this action in Neuroscience and Safety, but with an ambition to have broader applicability.
We have used complementary approaches to [1] identify those factors associated with increased robustness, rigour and validity of in vivo research; [2] develop a quality framework for individual studies and for research groups; [3] pilot that framework; [4] refine the quality framework based on these pilot studies; and [5] put in place arrangements for the sustainability of the framework.
Conclusions of the Action: The project management WP1 guided the consortium in the timely completion of Milestones and Deliverables in fulfilment of the consortium’s contractual duties and effective dissemination and communication activities. In WP2, the evaluation of published and consortium data across three common paradigms (Irwin, EEG, OFT) was challenging due to limited reporting and study compatibility, but we highlighted specific areas for improvement, such as reporting of measures to reduce the risk of bias and specific aspects of experimental design within each paradigm. In WP3 we developed guidance for researchers working with animal experiments to increase rigour in design, conduct and analysis, based on a systematic review of existing guidelines and prospectively tested for feasibility by project partners. In WP4 we conducted multicentre studies and showed that standardisation and heterogenisation of protocols (open field), detailed definition of qualitative endpoints (Irwin), and centralized data analysis (EEG) could each reduce between lab variation. Drawing from this experience, WP5 developed a Quality System (QS) and associated tools to support scientists conducting non-regulated preclinical research. These have been released for public use and serve as a basis for building a sustainable post-funding future. WP6 have articulated governance elements for the EQIPD QS. We have gone beyond simply exploring sustainability options by creating a non-profit follow up organisation, the Guarantors of EQIPD, which will provide oversight to organisations wishing to provide EQIPD QS assessments. We developed a web-based learning environment in WP7 to learning opportunities tailored to early career researchers working in industry or academia with formal learning provided in an E-learning course and summer school. In WP8 we developed a new ontology for metadata of in vivo preclinical neuroscience experiments and have secured arrangements for data availability beyond the project. WP9 provided ethical insights to and oversight of our activities.
We made good progress despite the challenges of the COVID pandemic. In WP2 we developed automation tools to accelerate systematic reviews; identified limited reporting of measures to reduce risks of bias and experimental deigns for the Open Field, Irwin and swEEG across published preclinical research; and that the variables and outcomes recorded for the Open Field, Irwin and swEEG data varied greatly between centres. We have made these automation tools available to the systematic review community. WP3 developed a framework for increasing rigour in design, conduct and analysis of animal experiments which is already informing training courses in in vivo research. WP4 showed that standardisation of protocols and clearer definition of outcome measures resulted in reduced variation between laboratories; and that for complex data sets (e.g. EEG, where there is site dependent variation in equipment and pre-processing steps) there may be benefit from centralized data analysis. WP5 defined the EQIPD QS with an associated framework to support implementation and maintenance, and that is described in an open-access publication in eLife (DOI: 10.7554/eLife.63294); developed the EQIPD Toolbox as a central repository of information related to good research practice (https://eqipd-toolbox.paasp.net/wiki/Toolbox); and co-developed (with stakeholders) targeted tools for self-assessment of existing research quality performance. WP6 conducted pilot deployments of the QS, and developed internal and external governance concepts including the establishment of the Guarantors of EQIPD. WP7 established an electronic training course on rigor and robustness in animal research (https://eqipd181379605.wordpress.com/) alongside an annual summer school for researchers. WP8 developed a new ontology for animal experiment metadata and have secured the future availability of EQIPD data.
Highlights include [1] the prospective organisation of multicentre studies with deliberately different degrees of alignment, showing the benefits of both standardisation and heterogenization; [2] the development of a quality system appropriate for academic labs; [3] a systematically derived set of guidelines for the conduct of animal research; and [5] the development of automation tools to assist systematic reviews at large scale.
In time this work will (1) improve European citizens’ health and well-being, (2) improve data quality of pre-clinical studies, (3) contribute to improved animal welfare: 3Rs + robustness, (4) enhance intellectual property protection and regulatory success, (5) support a cultural change in the implementation of quality principles in preclinical science, (6) build confidence between research partners, and (7) facilitate collaboration through common quality standards.
Socio-economic impact and the wider societal implications of the project: Some of these impacts are already apparent: The de-duplication tool developed in WP2 is being used by a H2020 SPRINT project evaluating the effects of plant protection products on the environment and human health (https://www.sprint-h2020.eu/) and in work with the European Food Safety Agency. The guidance for the conduct of animal research developed by WP3 and the work done in WP7 informed the curriculum for the Edinburgh Research Optimisation Course, launched in summer 2021. By providing richer guidance on the role of standardisation and heterogenization in multicentre animal experiments, our findings will contribute to greater validity in preclinical data. By providing a QS implemented in at the level of the laboratory we will contribute to the rigor and robustness of preclinical research, improving drug development for human disease. We expect that the use of the EQIPD framework will go beyond the original boundaries – both geographically (by expanding to countries not represented in the consortium) and by addressing the needs of other areas of science (e.g. plant biology, environmental health and safety). One impact of the pandemic was that we were compelled to consider alternatives to on site assessment visits, and this allowed us to create a lean assessment system without compromising the assessment of research data quality.
In time this work will (1) improve European citizens’ health and well-being, (2) improve data quality of pre-clinical studies, (3) contribute to improved animal welfare: 3Rs + robustness, (4) enhance intellectual property protection and regulatory success, (5) support a cultural change in the implementation of quality principles in preclinical science, (6) build confidence between research partners, and (7) facilitate collaboration through common quality standards.
Socio-economic impact and the wider societal implications of the project: Some of these impacts are already apparent: The de-duplication tool developed in WP2 is being used by a H2020 SPRINT project evaluating the effects of plant protection products on the environment and human health (https://www.sprint-h2020.eu/) and in work with the European Food Safety Agency. The guidance for the conduct of animal research developed by WP3 and the work done in WP7 informed the curriculum for the Edinburgh Research Optimisation Course, launched in summer 2021. By providing richer guidance on the role of standardisation and heterogenization in multicentre animal experiments, our findings will contribute to greater validity in preclinical data. By providing a QS implemented in at the level of the laboratory we will contribute to the rigor and robustness of preclinical research, improving drug development for human disease. We expect that the use of the EQIPD framework will go beyond the original boundaries – both geographically (by expanding to countries not represented in the consortium) and by addressing the needs of other areas of science (e.g. plant biology, environmental health and safety). One impact of the pandemic was that we were compelled to consider alternatives to on site assessment visits, and this allowed us to create a lean assessment system without compromising the assessment of research data quality.