Final Report Summary - ECLAT (European Cluster Assimilation Technolgy)
Executive Summary:
The European Cluster Assimilation Technology (ECLAT) project is funded by the European Commission through the Seventh Framework Programme (FP7), as part of the Exploitation of Space Science and Exploration Data theme. The project partners within ECLAT are the University of Leicester, UK, the Institutet för rymdfysik, Sweden, St. Petersberg State University, Russia, the Finnish Meteorological Institute, Finland, and the Oesterreichische Akademie der Wissenschaften, Austria.
The aim of the project was to provide contextual observations from space- and ground-based observatories and physics-based modelling in support of the European Space Agency’s Cluster magnetospheric science mission. The ECLAT data are distributed to the scientific community through the Cluster Active Archive. In addition to providing data, the project supported four scientific workshops dedicated to exploitation of the ECLAT data and dissemination of information regarding the project to the wider community.
In the lifetime of the project, 15 scientific papers were published, with many more submitted. Over 100 presentations were given at scientific workshops or conferences. Continued scientific output from the project is anticipated after the formal end of the project.
Project Context and Objectives:
The European Cluster Assimilation Technology (ECLAT) project was funded by the European Commission through the Seventh Framework Programme (FP7), as part of the Exploitation of Space Science and Exploration Data theme. The project partners within ECLAT are the University of Leicester, UK, the Institutet för rymdfysik, Sweden, St. Petersberg State University, Russia, the Finnish Meteorological Institute, Finland, and the Oesterreichische Akademie der Wissenschaften, Austria. As stated in the opening paragraph of the ECLAT proposal document, the aim of ECLAT was to
“...provide a novel and unique data base and tools for space scientists, by providing an upgrade of the European Space Agency’s Cluster Active Archive (CAA). The CAA is a state-of-the-art space plasma physics data repository, which will soon contain over 10 years of magnetospheric observations from the ESA Cluster multi-spacecraft mission. Although an invaluable resource to the space plasma physics community, the multi-instrument data are difficult to mine and analyze, and lack supporting contextual data which impedes scientific progress. The ECLAT programme will ingest into the CAA supporting data from other space- and ground-based observatories, provide data mining routines, refined data products and software tools for their visualization, and develop existing European magnetospheric modelling infrastructure to provide context for the observational data. Such an open-access, on-line resource will go beyond anything currently available in the space plasma physics community.”
The ECLAT proposal grew out of the activities of the Cluster and Double Star Ground-Based Working Group, of which Steve Milan and Hermann Opgenoorth have been coordinators. ECLAT aimed to bring a variety of ground-based and related observations to the general Cluster community in a form that was easy to analyze, visualize, and interpret, facilitating exploitation of the Cluster data-set.
In addition, the project was to support four scientific workshops that would act as impetus for exploitation of the data produced by ECLAT and also to disseminate information regarding the project to the wider scientific community. Each workshop was themed to a particular deliverable of the project, including magnetic field line mapping, meso-scale and global scale aspects of magnetospheric dynamics, and the assimilation of observations into physics-based models of the magnetosphere.
Project Results:
The ECLAT project was divided into 6 main work packages: WP100 – management, WP200 – in situ boundary and region identification, WP300 – meso-scale science, WP400 – global scale science, WP500 – physics-based modelling of the magnetosphere, and WP600 – science workshops and dissemination. These work-packages were further subdivided to provide specific deliverables, which are described below.
WP210 In situ data products: Cluster event list (D210.1)
Cluster in situ observations have been used to identify different three different magnetotail event types: magnetotail magnetic field dipolarization, large magnetic field gradient (current) events in the magnetotail (e.g. reconnection, strong field-aligned currents), and wavy current sheets. Separate event catalogues for each type are provided as the final data product containing event start and end epochs, as well as representative parameters of the in situ characteristics measured by Cluster. The input data for each event search algorithm is CAA Cluster data from tail season (July-October) periods 2001-2009. Full details can be found in the report associated with D210.1.
WP220 In situ data products: Cluster region and boundary identification (D220.1)
Cluster in situ observations, magnetic field and plasma data, have been investigated to find commonalities between differing magnetospheric regions and boundaries, including the magnetotail lobe, boundary region, outer plasma sheet, inner plasma sheet, and neutral sheet crossings. Software has been developed to identify such boundaries automatically in the data. This is necessary due to the extremely large amount of data produced by Cluster, which cannot all be analysed by hand. This deliverable has provided the times, locations, and characteristics of the regions, such that they can be analysed statistically, and to aid event identification. Full details can be found in the report associated with D220.1.
WP230 In situ data products: Event finding and visualization tool (D230.1)
This work package consists of the development, testing and refining of the search algorithms used to produce the ECLAT Cluster spacecraft in-situ data products in D210.1 and D220.1. Software was developed to visualize the boundary identifications from the Cluster in situ data. Details can be found in the report associated with D230.1.
WP310 Meso-scale data products: Equivalent current processing (D310.1)
MIRACLE comprises a network of ground-based magnetometers which measure the magnetic perturbations produced by currents flowing in the ionosphere overhead. Combining the observations from each of the stations allows the direction and the strength of these “equivalent currents” to be deduced. From the equivalent currents, it is possible to estimate the location and strength of field-aligned currents coupling between the ionosphere and the magnetosphere above. For the analysis of the magnetometer data to be successful, it is required that the background field – that not associated with perturbations from the currents – is known to high accuracy, and that diurnal “quiet-day” variations are removed. A significant aspect of this work package has concentrated on the characterization of these quiet-day curves. More details can be found in the report associated with D310.1.
WP320 Meso-scale data products: User interface and visualization tools (D320.1)
In concert with WP310, Matlab software has been developed to visualize the equivalent current data. More details can be found in the report associated with D320.1.
WP330 Cluster footprint mapping (D330.1 D330.2)
Magnetic field line tracing from the Cluster spacecraft location to the ionospheres in the Northern and Southern Hemispheres has been undertaken in the T96 (Tsyganenko, 1995) and TS05 (Tsyganenko and Sitnov, 2005) magnetic field models. Both models require knowledge of the concurrent solar wind and interplanetary magnetic field conditions, and this has been derived from the NASA OMNI data-base. The data have been converted into the required format for ingestion into the CAA (the Cluster Exchange Format, or CEF). Files exist for each year 2001-2009 and for each model (18 files). Software has been developed to visualize the Cluster footprint information in both geomagnetic and geographic coordinate systems; quick-look plots have been made for ingestion into the CAA, one for each day (3287 files). More details can be found in the report associated with D330.1.
WP410 Global-scale data products: SuperDARN (D410.1)
Observations from the radars comprising the SuperDARN network can be combined to produce maps of the ionospheric convection pattern at 2-min cadence. This has been undertaken for the period 2000-2010. The data have been converted into CEF format, two files for each day, one each for the northern and southern hemispheres (9036 files). Quick-look plots have been produced, one for each month (132 files), one for each day (4018 files), and one for each hour (96432 files), to allow easy browsing of the data. The hour quick-look plots also include the footprint information from D330.1. More details can be found in the report associated with D410.1.
WP420 Global-scale data products: IMAGE (D420.1)
The Imager for Magnetopause-to-Aurora Global Exploration (IMAGE) spacecraft orbited the Earth between 2000-2005, making observations of the auroras using the Far Ultraviolet (FUV) cameras. The IMAGE data have been mapped into the same grid as the SuperDARN data, to facilitate inter-comparison. The wideband imaging camera (WIC) and spectrographic imager (SI12) data have been converted to CEF format (3569 files). Quick-look plots have been produced, four for each month (104 files), two for each day (3570 files), and two for each hour (85680 files), to allow easy browsing of the data. The hour quick-look plots also include the footprint information from D330.1. More details can be found in the report associated with D420.1.
WP430 System level products (D430.1)
This work-package provides system level contextual data. A key deliverable are measures of the open magnetic flux content of the magnetosphere, determined from a) in situ measurements of the magnetic field strength in the magnetotail lobes by Cluster and b) from observations of the size of the polar cap (the dim region encircled by the auroral oval) from global auroral imagery provided by the IMAGE FUV instrument. This information is supplemented by solar wind observations and geomagnetic indices from the NASA OMNI database. More details can be found in the report associated with D430.1.
WP440 Global-scale visualization software (D440.1)
In concert with D410.1 D420.1 and D430.1 IDL software was developed to facilitate the plotting of the global scale datasets produced. More details can be found in the report associated with D440.1.
WP510 MHD Simulations: GUMICS library (D510.1)
The aim of this work package was to produce a library of model magnetospheres for different solar wind conditions and strengths and orientations of the interplanetary magnetic field, for different dipole tilt conditions, and for different phases of the solar cycle as measured by the F10.7 index. These models predict the structure of the magnetosphere under different interplanetary conditions. These models are then available to compare with the Cluster observations, and to provide a 3D context within which to view the Cluster observations. The runs are available on the Finnish Meteorological Institute website, as dictated in the description of work. More details regarding the GUMICS library runs can be found in the report associated with D510.1.
WP520 MHD Simulations: GUMICS year run and footprint mapping (D520.1)
Following on from WP510, the objective of WP520 was to run the GUMICS code for 12 months of the Cluster mission, using the measured solar wind and IMF from the NASA OMNI data set as input. It was decided that this should occur for the year 2002, based on the availability of SuperDARN and IMAGE data, and the fact that Cluster data coverage in the very early part of the mission was less than 100%. The full output of the simulation is available on the Finnish Meteorological Institute website, while quicklook plots are available through the CAA. More details can be found in the report associated with D520.1.
WP600 Science and Validation workshops (D600.1 D600.2 D600.3 D600.4 D610.1 D620.1 D630.1 D640.1)
Four ECLAT Science and Validation workshops were held between 2012 and 2014. These were attended by the ECLAT project partners, members of the Advisory Panel, and independent researchers, each workshop with a total attendance of approximately 30 people. Full meeting reports are available as D600.1 D600.2 D600.3 and D600.4. Separate reports on the assessment of the accuracy of mapping, meso-scale science, GUMICS reanalysis, and global scale science arising from the workshops are available as D610.1 D620.1 D630.1 and D640.1.
Potential Impact:
The outcome of the project has been the inclusion of several datasets into the Cluster Active Archive which enhance the science that comes from analysis of the Cluster data. An unfortunate situation within the space plasma physics community is that scientists are often biased towards one particular measurement type, for instance in situ particles measurements or ground-based radar measurements, and does not use all the resources available.
Partially this problem arises because many datasets are complex to obtain, analyse and plot, and their interpretation takes a lot of background knowledge. The ECLAT project aimed to add ground-based data to an archive of space-based observations to attempt to bridge this gap. The ethos has been to make the data as readily accessible and as easy to plot as possible, facilitating its use by the non-expert. The data are additionally be available in quick-look form, making event selection straight-forward.
In addition, additional auxiliary data sets have been produced to aid comparison between the space-based and ground-based observations: magnetic field-line tracing allows the spacecraft to be placed within the global context measured by the ground-based instrumentation; this information has not been readily available before.
Alongside the ground-based data, some of the Cluster data that is in the archive has been processed to provide “value-added” data products. These include the identification of crossings of important geophysical boundaries of the magnetosphere by the Cluster spacecraft. This facilitates statistical analysis of the Cluster data much, and simplifies event selection for case-by-case studies.
Finally, physics-based modelling of the magnetosphere for realistic solar wind conditions has provided a 3D map of the magnetosphere within which to place the Cluster observations in context. Magnetospheric modelling on the scale undertaken by ECLAT has not been attempted previously. This work required modifications and validation of the models, with dividends for future modelling efforts.
The ultimate aim of ECLAT was to improve the usability of the Cluster dataset, and to foster greater understanding and collaboration between scientists studying the magnetosphere from a ground-based and a space-based perspective.
List of Websites:
www.eclat-project.eu
The European Cluster Assimilation Technology (ECLAT) project is funded by the European Commission through the Seventh Framework Programme (FP7), as part of the Exploitation of Space Science and Exploration Data theme. The project partners within ECLAT are the University of Leicester, UK, the Institutet för rymdfysik, Sweden, St. Petersberg State University, Russia, the Finnish Meteorological Institute, Finland, and the Oesterreichische Akademie der Wissenschaften, Austria.
The aim of the project was to provide contextual observations from space- and ground-based observatories and physics-based modelling in support of the European Space Agency’s Cluster magnetospheric science mission. The ECLAT data are distributed to the scientific community through the Cluster Active Archive. In addition to providing data, the project supported four scientific workshops dedicated to exploitation of the ECLAT data and dissemination of information regarding the project to the wider community.
In the lifetime of the project, 15 scientific papers were published, with many more submitted. Over 100 presentations were given at scientific workshops or conferences. Continued scientific output from the project is anticipated after the formal end of the project.
Project Context and Objectives:
The European Cluster Assimilation Technology (ECLAT) project was funded by the European Commission through the Seventh Framework Programme (FP7), as part of the Exploitation of Space Science and Exploration Data theme. The project partners within ECLAT are the University of Leicester, UK, the Institutet för rymdfysik, Sweden, St. Petersberg State University, Russia, the Finnish Meteorological Institute, Finland, and the Oesterreichische Akademie der Wissenschaften, Austria. As stated in the opening paragraph of the ECLAT proposal document, the aim of ECLAT was to
“...provide a novel and unique data base and tools for space scientists, by providing an upgrade of the European Space Agency’s Cluster Active Archive (CAA). The CAA is a state-of-the-art space plasma physics data repository, which will soon contain over 10 years of magnetospheric observations from the ESA Cluster multi-spacecraft mission. Although an invaluable resource to the space plasma physics community, the multi-instrument data are difficult to mine and analyze, and lack supporting contextual data which impedes scientific progress. The ECLAT programme will ingest into the CAA supporting data from other space- and ground-based observatories, provide data mining routines, refined data products and software tools for their visualization, and develop existing European magnetospheric modelling infrastructure to provide context for the observational data. Such an open-access, on-line resource will go beyond anything currently available in the space plasma physics community.”
The ECLAT proposal grew out of the activities of the Cluster and Double Star Ground-Based Working Group, of which Steve Milan and Hermann Opgenoorth have been coordinators. ECLAT aimed to bring a variety of ground-based and related observations to the general Cluster community in a form that was easy to analyze, visualize, and interpret, facilitating exploitation of the Cluster data-set.
In addition, the project was to support four scientific workshops that would act as impetus for exploitation of the data produced by ECLAT and also to disseminate information regarding the project to the wider scientific community. Each workshop was themed to a particular deliverable of the project, including magnetic field line mapping, meso-scale and global scale aspects of magnetospheric dynamics, and the assimilation of observations into physics-based models of the magnetosphere.
Project Results:
The ECLAT project was divided into 6 main work packages: WP100 – management, WP200 – in situ boundary and region identification, WP300 – meso-scale science, WP400 – global scale science, WP500 – physics-based modelling of the magnetosphere, and WP600 – science workshops and dissemination. These work-packages were further subdivided to provide specific deliverables, which are described below.
WP210 In situ data products: Cluster event list (D210.1)
Cluster in situ observations have been used to identify different three different magnetotail event types: magnetotail magnetic field dipolarization, large magnetic field gradient (current) events in the magnetotail (e.g. reconnection, strong field-aligned currents), and wavy current sheets. Separate event catalogues for each type are provided as the final data product containing event start and end epochs, as well as representative parameters of the in situ characteristics measured by Cluster. The input data for each event search algorithm is CAA Cluster data from tail season (July-October) periods 2001-2009. Full details can be found in the report associated with D210.1.
WP220 In situ data products: Cluster region and boundary identification (D220.1)
Cluster in situ observations, magnetic field and plasma data, have been investigated to find commonalities between differing magnetospheric regions and boundaries, including the magnetotail lobe, boundary region, outer plasma sheet, inner plasma sheet, and neutral sheet crossings. Software has been developed to identify such boundaries automatically in the data. This is necessary due to the extremely large amount of data produced by Cluster, which cannot all be analysed by hand. This deliverable has provided the times, locations, and characteristics of the regions, such that they can be analysed statistically, and to aid event identification. Full details can be found in the report associated with D220.1.
WP230 In situ data products: Event finding and visualization tool (D230.1)
This work package consists of the development, testing and refining of the search algorithms used to produce the ECLAT Cluster spacecraft in-situ data products in D210.1 and D220.1. Software was developed to visualize the boundary identifications from the Cluster in situ data. Details can be found in the report associated with D230.1.
WP310 Meso-scale data products: Equivalent current processing (D310.1)
MIRACLE comprises a network of ground-based magnetometers which measure the magnetic perturbations produced by currents flowing in the ionosphere overhead. Combining the observations from each of the stations allows the direction and the strength of these “equivalent currents” to be deduced. From the equivalent currents, it is possible to estimate the location and strength of field-aligned currents coupling between the ionosphere and the magnetosphere above. For the analysis of the magnetometer data to be successful, it is required that the background field – that not associated with perturbations from the currents – is known to high accuracy, and that diurnal “quiet-day” variations are removed. A significant aspect of this work package has concentrated on the characterization of these quiet-day curves. More details can be found in the report associated with D310.1.
WP320 Meso-scale data products: User interface and visualization tools (D320.1)
In concert with WP310, Matlab software has been developed to visualize the equivalent current data. More details can be found in the report associated with D320.1.
WP330 Cluster footprint mapping (D330.1 D330.2)
Magnetic field line tracing from the Cluster spacecraft location to the ionospheres in the Northern and Southern Hemispheres has been undertaken in the T96 (Tsyganenko, 1995) and TS05 (Tsyganenko and Sitnov, 2005) magnetic field models. Both models require knowledge of the concurrent solar wind and interplanetary magnetic field conditions, and this has been derived from the NASA OMNI data-base. The data have been converted into the required format for ingestion into the CAA (the Cluster Exchange Format, or CEF). Files exist for each year 2001-2009 and for each model (18 files). Software has been developed to visualize the Cluster footprint information in both geomagnetic and geographic coordinate systems; quick-look plots have been made for ingestion into the CAA, one for each day (3287 files). More details can be found in the report associated with D330.1.
WP410 Global-scale data products: SuperDARN (D410.1)
Observations from the radars comprising the SuperDARN network can be combined to produce maps of the ionospheric convection pattern at 2-min cadence. This has been undertaken for the period 2000-2010. The data have been converted into CEF format, two files for each day, one each for the northern and southern hemispheres (9036 files). Quick-look plots have been produced, one for each month (132 files), one for each day (4018 files), and one for each hour (96432 files), to allow easy browsing of the data. The hour quick-look plots also include the footprint information from D330.1. More details can be found in the report associated with D410.1.
WP420 Global-scale data products: IMAGE (D420.1)
The Imager for Magnetopause-to-Aurora Global Exploration (IMAGE) spacecraft orbited the Earth between 2000-2005, making observations of the auroras using the Far Ultraviolet (FUV) cameras. The IMAGE data have been mapped into the same grid as the SuperDARN data, to facilitate inter-comparison. The wideband imaging camera (WIC) and spectrographic imager (SI12) data have been converted to CEF format (3569 files). Quick-look plots have been produced, four for each month (104 files), two for each day (3570 files), and two for each hour (85680 files), to allow easy browsing of the data. The hour quick-look plots also include the footprint information from D330.1. More details can be found in the report associated with D420.1.
WP430 System level products (D430.1)
This work-package provides system level contextual data. A key deliverable are measures of the open magnetic flux content of the magnetosphere, determined from a) in situ measurements of the magnetic field strength in the magnetotail lobes by Cluster and b) from observations of the size of the polar cap (the dim region encircled by the auroral oval) from global auroral imagery provided by the IMAGE FUV instrument. This information is supplemented by solar wind observations and geomagnetic indices from the NASA OMNI database. More details can be found in the report associated with D430.1.
WP440 Global-scale visualization software (D440.1)
In concert with D410.1 D420.1 and D430.1 IDL software was developed to facilitate the plotting of the global scale datasets produced. More details can be found in the report associated with D440.1.
WP510 MHD Simulations: GUMICS library (D510.1)
The aim of this work package was to produce a library of model magnetospheres for different solar wind conditions and strengths and orientations of the interplanetary magnetic field, for different dipole tilt conditions, and for different phases of the solar cycle as measured by the F10.7 index. These models predict the structure of the magnetosphere under different interplanetary conditions. These models are then available to compare with the Cluster observations, and to provide a 3D context within which to view the Cluster observations. The runs are available on the Finnish Meteorological Institute website, as dictated in the description of work. More details regarding the GUMICS library runs can be found in the report associated with D510.1.
WP520 MHD Simulations: GUMICS year run and footprint mapping (D520.1)
Following on from WP510, the objective of WP520 was to run the GUMICS code for 12 months of the Cluster mission, using the measured solar wind and IMF from the NASA OMNI data set as input. It was decided that this should occur for the year 2002, based on the availability of SuperDARN and IMAGE data, and the fact that Cluster data coverage in the very early part of the mission was less than 100%. The full output of the simulation is available on the Finnish Meteorological Institute website, while quicklook plots are available through the CAA. More details can be found in the report associated with D520.1.
WP600 Science and Validation workshops (D600.1 D600.2 D600.3 D600.4 D610.1 D620.1 D630.1 D640.1)
Four ECLAT Science and Validation workshops were held between 2012 and 2014. These were attended by the ECLAT project partners, members of the Advisory Panel, and independent researchers, each workshop with a total attendance of approximately 30 people. Full meeting reports are available as D600.1 D600.2 D600.3 and D600.4. Separate reports on the assessment of the accuracy of mapping, meso-scale science, GUMICS reanalysis, and global scale science arising from the workshops are available as D610.1 D620.1 D630.1 and D640.1.
Potential Impact:
The outcome of the project has been the inclusion of several datasets into the Cluster Active Archive which enhance the science that comes from analysis of the Cluster data. An unfortunate situation within the space plasma physics community is that scientists are often biased towards one particular measurement type, for instance in situ particles measurements or ground-based radar measurements, and does not use all the resources available.
Partially this problem arises because many datasets are complex to obtain, analyse and plot, and their interpretation takes a lot of background knowledge. The ECLAT project aimed to add ground-based data to an archive of space-based observations to attempt to bridge this gap. The ethos has been to make the data as readily accessible and as easy to plot as possible, facilitating its use by the non-expert. The data are additionally be available in quick-look form, making event selection straight-forward.
In addition, additional auxiliary data sets have been produced to aid comparison between the space-based and ground-based observations: magnetic field-line tracing allows the spacecraft to be placed within the global context measured by the ground-based instrumentation; this information has not been readily available before.
Alongside the ground-based data, some of the Cluster data that is in the archive has been processed to provide “value-added” data products. These include the identification of crossings of important geophysical boundaries of the magnetosphere by the Cluster spacecraft. This facilitates statistical analysis of the Cluster data much, and simplifies event selection for case-by-case studies.
Finally, physics-based modelling of the magnetosphere for realistic solar wind conditions has provided a 3D map of the magnetosphere within which to place the Cluster observations in context. Magnetospheric modelling on the scale undertaken by ECLAT has not been attempted previously. This work required modifications and validation of the models, with dividends for future modelling efforts.
The ultimate aim of ECLAT was to improve the usability of the Cluster dataset, and to foster greater understanding and collaboration between scientists studying the magnetosphere from a ground-based and a space-based perspective.
List of Websites:
www.eclat-project.eu