Final Report Summary - IMPEX (Integrated Medium for Planetary Exploration)
Executive Summary:
IMPEx was a four year collaborative project, funded under the 7th framework of the European Union (FP7) that started in June 2011. Its consortium is made up of four members, scientific institutions from Austria (IWF-OeAW, the coordinating entity), Finland (FMI), France (CNRS), and Russia (SINP). The projects main goal is the creation of a software environment that facilitates and supports the comparison and overlay of modelled space plasma data vis-à-vis real world observational data, obtained in the course of past and future space missions. The software environment is consisting of several well established tools (mainly CDPP-AMDA, CDPP-3DView and CLWeb) provided by CNRS, for the analysis and visualization of space mission data. These tools have been extended to handle various aspects of simulation data with the goal of gaining a better understanding of observed phenomena, as well as to optimize existing simulation codes and related models. IMPEx enabled tools are aimed at supporting mission planners and spacecraft designers, by providing modelled and empirical data for e.g. virtual space flights in simulated planetary (plasma) environments.
During the first project year the concepts and goals outlined in the proposal were concretized in the course of the requirements definition (supported by the creation and analysis of scientific use cases) and architectural design phase (see D2.2 and D2.3). Several approaches were discussed and finally agreed upon and carefully documented. The main pillars of the IMPEx system were created; the IMPEx Protocol and the IMPEx Data Model basically cover all requirements from a technical and conceptual perspective, and allow for the exchange of data between various nodes of the topology as captured in the IMPEx Configuration. In the second and third year of the project the focus was primarily set on the finalization of all designs and of course the implementation, including the preparation of test cases for testing and system validation. In this period the project’s advisory boards were an important source of feedback and orientation for the team, with the aim to provide a system that is relevant and capable of solving real life problems in the course of scientific investigations. The boards are also pivotal in capturing all relevant requirements for the IMPEx Portal that was developed at IWF-OeAW during the second half of the project. It provides a one-stop-solution for anyone who is interested in IMPEx and wants to learn about the system through a hands-on experience, covering all aspects of IMPEx and integrating all tools of the environment.
The final project year was dedicated to the finalization of remaining definitions, features and functionalities deemed essential by the user community, as well as an integrated test effort to assure the required stability and general quality of the system. Further the IMPEx Data Model became an official part of the SPASE data model, upon which it is based on. The IMPEx Data Model hence becomes the SPASE simulation extension that will enable all users and applications of SPASE to integrate simulation data into their (database) systems, being able to handle and search simulation data as it is also possible for observational data.
The IMPEx website was established very early on during the project and provides comprehensive access to all relevant information (technically and scientifically) in order to gain a detailed insight into IMPEx, its tools and capabilities. The website offers a rich selection of video tutorials, comprehensive online material as well as a dedicated project podcast, featuring team members, members of the advisory boards etc. in 16 episodes, covering a wide range of subjects about and surrounding IMPEx. All in all it can be said that IMPEx exceeded expectations by its protagonists, in particular by setting a de facto standard in the field of simulation data and by establishing a prototype system that is based on common design principles of the Virtual Observatory (VO) and planetary science community.
Project Context and Objectives:
Europe as a leading participant in international endeavours (e.g. Rosetta, Mars Express, Venus Express, ESA/NASA Cassini-Huygens) to explore our solar system and planetary objects is also at the forefront in the development of theoretical models to simulate conditions in space. Empirical data derived via actual measurements during space missions and data obtained through computational model runs are two major aspects of modern space research. While it is obvious that models can only be further developed by comparing the results to real word measurements, it is as important to then use these models (once sufficiently confirmed by observational data) to interpret new measurements and to understand the physical processes behind the data. Both processes need adequate software support that was practically not available in planetary science by the time the project idea of IMPEx was developed. Once realized, modelled data can also be easily applied to support spacecraft designers and engineering in general, by providing the means to simulate conditions in planetary environments to e.g. obtain accurate requirements for spacecraft hardening, and design requirements in general. The last major application is to fill measurement gaps in observational data using modelled data derived via validated (sic) models.
Figure 1: The IMPEx environment with its main tools and SMDBs, connected through the IMPEx Protocol that is fully implemented by the IMPEx Portal. All data is stored compliant with the IMPEx Data Model and can be easily exchanged between the various nodes (tools and SMDBs).
The consortium of IMPEx includes several institutions that have developed world renowned computational models in the field of space plasma physics, being able to simulate the (plasma) environments of the majority of planetary objects in the solar system, including moons and comets. The IMPEx modelling sector is comprised of two hybrid models provided by FMI and LATMOS (CNRS), a magneto hydrodynamic model by FMI, as well as a unique paraboloid model provided by SINP – the models in detail are:
- The worldwide recognized 3D hybrid modelling platform HYB for study of Solar System objects’ plasma environments (developed and hosted at FMI)
- Global MHD modelling platform for 3D terrestrial magnetosphere (developed and hosted at FMI)
- The global 3D Paraboloid Magnetospheric Model for simulation of magnetospheres of different Solar System objects (developed and hosted at SINP). This model also offers on-demand calculations (see IMPEx Protocol), i.e. responses are more or less real-time.
- The LATMOS hybrid model developed and hosted at Université de Versailles Saint-Quentin.
Further, several web based analysis tools (CDPP-AMDA, CLWeb, the Java based visualization tool CDPP-3DView, see Figure 2) and user interfaces for the simulation databases (SMDBs) provided by FMI, SINP and LATMOS are part of the environment. The user interfaces of FMI and SINP also offer WebGL based visualization capabilities (Figure 1).
Figure 2: Visualization of a magnetic field simulation run interpolated onto the orbit of Mars Express, visualized via the Java based 3D tool CDPP-3DView provided by CDPP. The tool has been extended with regard to visualization capabilities required for the visualization of magnetic field and plasma data and enabled to communicate with the IMPEx environment (IMPEx enabled tool).
One of the main technical objectives of the project was to enable these tools to exchange data with all participating SMDBs and hence integrate data obtained through complex simulation runs with observational data that is already being processed and analysed by the aforementioned tools (the available data was also significantly increased in the course of IMPEx, see D2.13). Another (scientific) objective is to further develop models in particular the paraboloid models for Mercury, Jupiter, Saturn and Earth by SINP (also see D4.4 D4.5 and D4.6).
Connecting tools and databases is a challenging task, in particular when complex scientific data is involved. Numerous problems must be tackled as e.g. the aspect of generality, i.e. the ability to scale and reuse the solutions defined and implemented in other systems, and to extend the “data network” far beyond the original set of software applications and data collections.
In this sense the approach of IMPEx is highly compatible with the vision of the International Virtual Observatory Alliance (IVOA) that calls for astronomical datasets and other resources to work as a seamless whole. The philosophy followed here is that it is unproductive to put effort in “reinventing the wheel”, i.e. to implement similar functionalities over and over again in different tools. Instead the IVOA initiative, as does IMPEx, aims at connecting systems in order to be able to easily exchange (scientific) data and leverage functionalities of a rich set of tools. Thus, IMPEx uses several standards that originated out of the VO community that were defined by IVOA. Among them is the VOTable format that is heavily used in IMPEx, to exchange trajectories etc. SAMP is another IVOA standard used that allows web based tools to communicate with each other in a straight forward way, circumventing the back ends.
Besides scientific and technological aspects, there is also a strong relation to public education as well as a significant socio economic component in general. IMPEx delivered a wide range of materials that can be used in public education at the level of public schools as well as higher education including university level courses (see D2.14 D2.15 D3.12 D4.7 and D5.2). The website, which has been a central element for public outreach as well as an essential hub for technical information, features a number of tutorials, videos and extensive documentation of tools, models and the scientific approaches that lie underneath the environment. It is the hope that these materials will help to encourage young students to seek careers in space science and to engage in exciting research that is conducted in Europe and all over the world. In addition to that, the IMPEx Data Model (see Figure 37) provides a pivotal prerequisite in order to foster the exchange of scientific data, derived from computational models.
The IMPEx Data Model is, and will be even more so in the future, helping to bring the data and models outside of mission teams and specialized modelling groups, making them accessible and useful for the broad planetary science community, and, in this way, promoting the contribution of space assets to scientific and technological knowledge and development. Existing levels of scientific data exploitation, cooperation, as well as technical capabilities for communication of researchers including access to remote databases and computing infrastructures, provide ideal conditions for the integration of a combined modelling and data environment, as IMPEx is representing today.
As already briefly mentioned in Chapter 1.1 the IMPEx Data Model became the SPASE simulation extension and hence an official part of SPASE, which is one of the leading and most widely applied data models in space plasma physics worldwide. By integrating the IMPEx definitions, SPASE now is capable to also reflect on plasma data that was derived by computational modelling, and hence becomes the ideal environment for a combined use of both types of data. Since the IMPEx simulation extensions are conceptually close to the corresponding data structure and definitions for empirical plasma data, it is now a straight forward task to match data sets from both worlds. To use a practical example, as we can see in Figure 37, NumericalOutput is related to a certain Repository just as is NumericalData used in “traditional” SPASE. However, in the context of the simulation extensions the ontology is adapted, and the Instrument element used for empirical data becomes the SimulationModel, to clearly specify the technical source of the data so it can be adequately assessed.
With the development of the IMPEx Protocol a commonly used approach in IT development has been followed, and a set of SOAP web services was defined, implemented and offered (via *.wsdl files) to the worldwide community, in order to enable it to fully leverage functionalities of the IMPEx environment. The idea is that through the provision of a common mechanism to expose modelled data including functionalities (i.e. web services) that can be applied on the data, also other SMDBs and infrastructures worldwide are encouraged to join the community and integrate their data, ever increasing the scientific scope of IMPEx. In mid-2015 there are already advanced prototype implementations available, and data from UCLA has e.g. been added to the IMPEx Configuration. There are many other projects (besides SPASE) and initiatives that benefit from IMPEx developments and vice versa that are cooperating with the project as e.g. EFTLA/Astronet, CCMC, VESPA - see D5.3 for further information.
New requirements coming up in the future can be addressed straight forward through further updates of the IMPEx Protocol by either adding new methods, extending existing methods (preserving backward compatibility) or by extending the IMPEx Data Model to be able to describe an even broader range of simulation data and models. This way IMPEx is to be regarded as a living definition that will grow along with its applications and future requirements.
New horizons could also be reached by integrating cloud computing capabilities in possible follow up projects. Already within IMPEx FMI has done some experimentation with cloud technologies, by deploying the HYB modelling code on cloud resources, in order to test integration of these technologies in possible follow-up projects. Since modelling runs are a vital part of IMPEx, the inclusion of cloud resources would also enable smaller institutions to do complex runs on-demand, tailored to specific empirical data, to be analysed. All in all it can be said, that the modular approach of IMPEx lends itself perfectly to future extensions towards cloud computing and related technologies.
Project Results:
This chapter details the main results in science and technology for every RTD related work package of IMPEx as well as WP5 that was leading the implementation of the IMPEx Portal.
Data and Models Environment (DaME)
The major goal of Work Package 2 consisted in providing a user-friendly environment where the models and their results can be compared with the data of space missions and used jointly for scientific research and technological applications. The key objectives were:
• To design and implement an integrated environment of tools enabling the visualization of data originating from simulations, observations and models.
• To define and use standards for data and time table exchange between tools (CDPP-AMDA, CDPP-3DView, CLWeb and simulation databases).
• To define and use several methods of data discovery, aimed at helping the user fine-tuning the observations/models/simulations comparison.
• To provide a variety of data representations in multiple dimensions.
• To initiate collaborations with similar infrastructures in order to avoid duplication of efforts.
Integrated environment of tools
The first task of WP2 was the definition of an architecture facilitating the communication between scientific software tools, computational services and databases, as depicted in the figure below.
Figure 3: Overview of the IMPEx architecture and flow of information.
The implementation was facilitated by the adoption of widely used standards like SOAP, REST and HTTP, or IVOA SAMP and VOTable. In order to exchange data sets, a detailed description of their contents is necessary. This is achieved with the definition of metadata. Since a metadata model for observations already existed, it has been extended to take into account data coming from simulations. This task was conducted in collaboration with WP3.
Interfaces and protocols for data and time tables exchange
It is now possible to browse the contents of PMM and HMM catalogues in CDPP-3DView, CDPP-AMDA and CLWeb, using a file containing all the metadata necessary to access HMM or PMM data. A common format has been defined for this file, which is provided by all simulation databases (SMDBs). Every IMPEx tool must parse this file to display an interface for searching and getting data.
It is also possible to exchange data between CDPP-AMDA and CDPP-3DView using the SAMP protocol, widely used by astronomers. When CDPP-AMDA and CDPP-3DView are active on the user’s desktop, CDPP-AMDA can send data in VOTable format via a SAMP-Hub. These data can be scalars, vectors or spectrograms. CDPP-3DView implements a SAMP listener, which is waiting on a signal from CDPP-AMDA. When data are available in the SAMP-Hub, CDPP-3DView creates the related graphical objects and displays them in the scene. If these data are not in the same coordinate system, a transformation of coordinate system is performed.
Figure 4: Browsing IMPEx data in CDPP-3DView (with filters).
New methods for data discovery
New methods for data discovery were investigated and implemented; the selection of a run for comparison with observations is composed of two steps:
1) The first step is via the HMM and/or PMM simulation and model databases (SMDBs) catalogues browsing interface, implemented in the accessing interface. The files containing the metadata are provided by the SMDBs; currently FMI, LATMOS and SINP. They are parsed and displayed entirely as a hierarchy of data.
2) The second step consists in “filtering” the hierarchy by applying one or several criteria to the simulation input parameters contained in the metadata files, to reduce the size of the hierarchy of data, according to these criteria, which are selected by the user. An interface enabling this feature is provided in CDPP-3DView. It is e.g. possible in CDPP-3DView to shorten the list of runs to those related to a specific body (e.g. Mars or Earth).
The other way, an analytic (minimization) method of best fitting run among those available in HMM and GUMICS archives, has been investigated and implemented. This ordering method is called “N-index”. An SMDB that archives simulation runs, calculates this N-index and provides it through a web-service. This method, described below, is called getMostRelevantRun.
The best use of this method in the IMPEx infrastructure is to select data from GUMICS runs in CDPP-AMDA which already provides users with search and filtering capabilities in its workspace, but these capabilities are not applicable for a huge number of items, as it is the case for FMI/GUMICS, which provides about one hundred thousand runs.
Figure 5: Find the most relevant GUMICS Run in CDPP-AMDA.
Data representation in multiple dimensions
One of the great achievements of IMPEx is the possibility to display new types of scientific data, coming from observations as well as simulations in CDPP-3DView. Users can now display time series along the trajectory of spacecraft, field lines or flow lines, and cuts in two dimensions. Several use cases using these new capabilities in CDPP-3DView are presented below, including the comparison of simulated and observed time series, the synchronization of 2D and 3D displays, 2D-cuts, magnetic field lines, velocity flow lines, and spectrograms.
Figure 6: CDPP-3DView - Magnetic field vector and field lines around Saturn simulated by SINP, overlaid with CASSINI/MAG data, provided by AMDA (source PDS).
Figure 7: Impact of a CME (2012/06/16) visualized with in-situ data in CDPP-3DView. Cluster 1 and Geotail from CDPP-AMDA, simulations from FMI-GUMICS.
Figure 8: Display of several types of simulation data along Rosetta orbit in CDPP-3DView (hybrid model from FMI for comets).
Figure 9: Display in CDPP-3DView of several types of simulation data around Mars (MEX). Hybrid model from LATMOS.
Figure 10: Spectrogram of simulation data from FMI along Venus Express trajectory.
In CDPP-AMDA, it is now possible to display on the same 2D plot, time series of local or remote data coming from observations and remote data resulting of simulations, as depicted in the figure below:
Figure 11: CDPP-AMDA plot displaying comparison between observed data in the Martian environment (by the MEX spacecraft) and simulated ones (panels without the MEX tag). Magnetic field (6th panel from top) is absent from MEX instrumentation and can only be inferred from simulation models.
Collaboration with similar infrastructures
IMPEx WP2 has initiated several collaborations with similar infrastructures:
CCMC: The goal of the IMPEx-CCMC prototype was to render CCMC results more easily accessible to the wider community, by providing access, visualization and analysis via CDPP tools (CDPP-AMDA and CDPP-3DView). Another goal was to check whether the IMPEx Data Model, used to describe simulations to be compared with observations, was applicable to SMDBs that did not participate in IMPEx. The prototype was limited to one type of simulation, namely BATSRUS (MHD), and one run with data interpolated along the trajectory of several magnetospheric spacecraft. It focused mainly on access from CDPP-AMDA and CDPP-3DView. CCMC provides interpolation (in the 3D box) of physical quantities (fields and plasma parameters) along spacecraft trajectory as time series. They can now be directly compared to in-situ observations in CDPP-AMDA or CDPP-3DView.
EuroPlaNet-RI: An interface between IMPEx and EuroPlaNet-RI, another FP7 partner has been implemented, via a protocol defined by EuroPlaNet-RI, to access an IMPEx related database (CDPP-AMDA) from an external tool (TopCat). EuroPlaNet-RI has defined a common data model, EPNCore and a protocol, EPNTap. Both are used to give access to planetary science data. IMPEx has developed an interface between one of the participating tools, CDPP-AMDA and a search client designed and implemented by VO-Paris, in the framework of EuroPlaNet-RI. The same interface may be used between CDPP-AMDA and TopCat, a tool developed in the framework of IVOA (International Virtual Observatory Alliance) to analyze scientific data. These interfaces are based on EPNCore and EPNTap. In addition to that, IMPEx has created a translation mechanism between the IMPEx data model, which is based on SPASE and EPNCore. Every resource described within the IMPEx metadata trees can then be consumed by the EPN search client or by TopCat. Scientists familiar with TopCat are now able to use its numerous options to efficiently analyze and display a lot of datasets from space missions provided by CDPP-AMDA and simulations provided by IMPEx.
Extension of the CDPP-AMDA database of observations
A lot of datasets from space missions, relevant for observations/simulations comparisons, were added to the CDPP-AMDA database during IMPEx. They include data from missions to Mercury (Messenger), Venus (Venus Express, PVO), Mars (MAVEN, MGS, Mars Express), the Moon (Artemis) or the Giant planets (CASSINI, GALILEO, VOYAGER 1 & 2, PIONEER 10 & 11, ULYSSES), Comet Churyumov-Gerasimenko (Rosetta), and Comet Halley (Giotto).
Links to remote databases were also established for Venus (Venus Express) and the Moon (Artemis).
Outreach and educational products
A tutorial, demonstrating the access to data coming from observations and simulations with CDPP-AMDA and CDPP-3DView has been provided (see D2.15 as well as the demonstrator section of the IMPEx website, https://sites.google.com/site/impexfp7/). With this step-by-step guide users are able to visualize data related to the Earth magnetosphere, coming from several origins:
• Observations from CDPP-AMDA database
• SINP paraboloid model for the magnetic field
• FMI MHD model (GUMICS code)
• BATSRUS (MHD) run at CCMC
Hybrid and MHD Models (HMM)
WP3 (Hybrid and MHD Models (HMM)) was one of the two work packages which formed the IMPEx modelling sector. The basic objective of WP3 was to enable IMPEx to access modelling codes for simulation of plasma environments around planetary objects. Further it included the preparation of efficient interfaces for the operation of these codes and to work with their results. The modelling core of WP3 consisted of FMI’s hybrid (HYB) and magnetohydrodynamic (GUMICS) models and LATMOS’ hybrid model. Hybrid models were used for the study of solar system objects environments and the magnetohydrodynamic model for the terrestrial magnetosphere. The modelling infrastructure of WP3 builds on computational resources operated at FMI, as well as their related data bases and services.
The computational modelling activity in WP3 fed directly into WP2 in a way that the models and their data outputs were linked to the IMPEx infrastructure, which also included a large archive of space mission data (CDPP-AMDA) and a 3D visualization tool (CDPP-3DView).
The key products obtained in WP3 are the following:
• Defining the content and the structure of the IMPEx methods (*.wsdl files) and the structure of the simulation metadata (tree.xml files).
• IMPEx web service interfaces, which enable direct access to SMDB and, therefore, combining observation and simulation data.
• Web interface Hybrid Web Archive (HWA), which provides a user friendly interactive web access to FMI’s HYB and GUMICS model results and to its 3DHWA visualization facilities.
• Web interface LatHyS, which provides a user friendly interactive web access to LATMOS’ hybrid model runs results and to its visualization facilities and analysis tools.
• Open source IMPEx method scripts, which provide an example of how IMPEx methods have been implemented by FMI. These scripts help other model providers to build own
IMPEx compatible web services.
• Open source HYB model. This enables users to make own runs with the FMI’s HYB model and, importantly, check rationale and correctness of the used algorithms.
• 3D HWA visualization tool, 3DHWA, which helps FMI’s HMM SMDB users and users of the IMPEx methods to analyse runs in the HWA.
• Open source MATLAB scripts, provided by FMI, which provides MATLAB users an easy usage of the IMPEx methods
• Creating HMM modelling demonstrators for public education
The IMPEx Data Model, Interfaces, protocols and catalogues
A large effort on the definition and the implementation of the IMPEx Data Model has been conducted within WP3, together with WP2 and WP4. This effort has been consolidated in a new deliverable D3.14 not originally planned. Most of the development work for this data model concerned the description of the simulation runs and models. For this, we created new resources: SimulationModel, SimulationRun, SimulatedRegion, SimulationProduct, SimulationDomain, SimulationTime, and SimulationType (also see Figure 37).
We also made use of xml elements already defined in SPASE, to describe the simulation temporal and spatial parameters, as well as the targeted celestial body of the simulation. In all our developments we tried to limit the introduction of new terms or elements to facilitate the parsing of the metadata by automated tools (the larger the number of terms is, the more difficult it is for tools to handle them). An example of these new resources is presented in Figure 12.
Figure 12: Graphical representation of the SimulationModel resource
HMM environment data for CDPP-AMDA database
This effort consisted in linkage of HMM models and input/output data of modelling runs to the CDPP-AMDA database - it was performed in coordination with WP2. The developed data model link provides access to various modelling tools and data products.
The work has contributed to the definition of the tree.xml file which describes all HMM simulation runs and the accessible variables thereof. The file is publicly available from IMPEx’s data services.
Each simulation provider is responsible for providing the tree.xml on its server. A data model (standardized xml vocabulary and grammar) has been defined to standardize the tree.xml format. The tree.xml file describes run inputs and all simulation data, and provides links to the pre-computed data.
The IMPEx Simulation Data Model was designed as an extension of the existing SPASE Data Model, which is widely used for space physics databases. This compliance with standards ensures an integration of simulation databases in IMPEx, along with observation databases and an easier expendability of the IMPEx framework.
All methods have been defined for the SMDB. In HMM the methods are the following:
1. getDataPointValue: Returns a physical variable or variables at given point(s) from a model database.
2. getDataPointValueSpacecraft: Returns a physical variable or variables along the orbit of a spacecraft from an SMDB.
3. getDataPointSpectra: Returns the energy spectra at given point(s) from an SMDB.
4. getDataPointSpectraSpacecraft: Returns the energy spectra along the orbit of a spacecraft from an SMDB.
5. getSurface: Returns a physical variable or variables on a plane which normally is along the coordinate axis of an SMDB.
6. getFieldLine: Makes vector field tracing as e.g. magnetic field line tracing or flow line tracing.
7. getParticleTrajectory: Makes particle tracing (provided only by FMI).
8. getMostRelevantRun: Returns an ordered list of runs that have input values as close as possible to the provided set of values.
9. isAlive: returns the status of the database (Alive or Down).
One important task was to add ~1e5 GUMICS Earth magnetosphere result files into the tree.xml file in a way that keeps the size of the xml-file reasonably small and hence fast to use. The effort was successful and the method can be used efficiently on even such a large number of files. This also demonstrates that IMPEx methods and protocols are scalable and can handle in the future hundreds of thousands of missions and simulation result files. The HYB model was used to provide simulation results for Venus, Mars and comet 67P/Churymov-Gerashimenko. It further supports data analysis of ESA’s Venus Express, Mars Express and Rosetta missions, respectively.
HMM environment data for visualization
This task provided the outputs of HYB and MHD models for the CDPP-3DView visualization software provided by WP2. It is performed in coordination with WP2, and is connected to WP3.
The available data products and computational methods to access HMM simulation runs from CDPP-3DView were defined in the tree.xml and methods.xml files.
In addition to the CDPP-3DView visualization, a purely web based 3D visualization tool 3DHWA, was developed to provide a 3D tool to visualize HYB and GUMICS simulation runs in HWA. User picks up an interesting run from the HWA in a normal way. The user can then pick up plotting option “3D” (Figure 13) which enables interactive web based 3D analysis of the data.
Figure 13: An example of how 3D plots can be visualized via HWA (http://hwa.fmi.fi/) by its 3D visualization tool 3DHWA. The user can choose the “3D”page from the “Data Options”.
After that, a user can choose the analysed 3D region and how many planes are plotted.
Moreover, the user can add the orbit of the spacecraft at a given time range. The orbit is obtained by a web service provided by CDPP-AMDA (see Figure 14).
Figure 14: An example of a 3D plot obtained at HWA (http://hwa.fmi.fi/) by its 3D visualization tool 3DHWafter the user has pressed the button “Plot Data”, as shown in Fig. 13. The window makes it possible for a user to analyse data at various planes and to study how the data is related to the orbit of the spacecraft.
HMM-WWW interface for external users and infrastructures
The goal was to provide the HMM modelling tools, being further developed within IMPEx, for those users who operate data sets and services that are not part of the IMPEx core infrastructure (e.g. Europlanet-RI, HELIO, CCMC etc.), as well as for users who may want to apply HMM for basic research or for technological development.
Simulation runs stored in HWA can be accessed and searched by users. In Figure 15 an example is shown, where simulation runs are categorized according to the object and simulation run name - the physical and numerical parameters are also shown. Plot options and data download functionalities provide access to the simulation run results.
Simulation runs on HWA can also be searched by their input parameters - Figure 16 shows an example of that.
An example of access to simulation results (HWA’s 2D plotting facility) is demonstrated in Figure 17 (left). The variables to select for downloading are shown in Figure 17 (right). As noted before, in addition to 2D plots the HWA offers also a possibility for 3D plots.
The HMM-Direct Access Mechanism (HMM-DA) is realized using web service technology which is a standardized method of communication between two computers/applications over the internet. The interface to the service is described in a machine-processable format called WSDL (Web Services Description Language), which is an xml language. The interface defines all services (methods) that the server provides along with all necessary input and output format descriptions. Other systems interact with the web service by exchanging messages using the SOAP protocol.
HMM modelling demonstrators for public education
This effort included activities to provide a set of HMM modelling demonstrators, organized in the form of interactive web based educational services addressed to general public. The modelling demonstrators provide tutorials and explanations of the computational and physical backgrounds, making them ready for use in public educational institutions (schools, universities, planetariums, etc.).
Descriptions of the HYB modelling platform are available as a file and via the IMPEx Website (see https://sites.google.com/site/impexfp7/home/hmm). Furthermore, as illustrated in Figure 18, the HWA main page (http://hwa.fmi.fi) contains several links to the webpage which has been prepared for HMM modelling demonstrators for public education purposes. The pages describe the HYB (Figure 18 b) and GUMICS (Figure 18 c) models, the physical basics of the models, example of results, publications etc. Moreover, several videos were made and distributed via YouTube which describe and demonstrate the properties of the HWA in English as well as Finnish language, including code and usage of Matlab scripts. Last but not least, the HYB code, Matlab and FMI’s IMPEx web service scripts are made openly available at GitHub (see https://github.com/fmihpc/impex-tools).
Figure 18: An illustration of the access of video demonstrations and descriptions of the models. The HWA main page a contains links to the description of the HYB model (b), GUMICS model (c) and YouTube demonstration videos about the HWA and open source MATLAB scripts (d).
Complementary catalog at LATMOS
The goal of this effort (also see task 3.7) was to complete the catalogue of hybrid simulation runs by creating a new SMDB and linking it to CDPP-AMDA and CDPP-3DView tools.
In addition to the development of the infrastructure of simulation databases and services, LATMOS has made a significant contribution to the definition of the IMPEx Data Model (see D3.14).
Figure 19: Screenshot of the LATMOS SMDB web interface.
The simulation catalogue is publically accessible and some documentation concerning the simulation model and the web services can be found on the LATMOS website (http://impex.latmos.ipsl.fr) - a screenshot showing a partial description of the simulation runs available is presented in Figure 19.
The catalogue includes simulation runs for various celestial bodies such as Mars, Mercury and Ganymede. This database is characterized by the provision (via open access) of 3D data cubes and the full set of data products, in a way that enables the whole community to access the various simulated quantities. The full plasma package released in the catalogue consists in electric and magnetic fields, electron number density and temperature, plasma bulk speed and all ion species information (density, velocity and temperature). A detailed description of the data-products is provided in D3.5. Information concerning the URL of these pre-calculated files is indicated in the tree.xml file. Tools like CDPP-AMDA or CDPP-3DView parsing the tree.xml have the information about the pre-calculated files (see D3.7).
Interoperability of the LatHyS website with VO-tools
In addition to the archive of full 3D cubes simulations results, we archive and provide pre-computed 2D plane cuts for all data products. These pre-computed products can also be downloaded; it gives a preview of the simulation results. Moreover, LATMOS web-interface proposes connection to VO tools, like Topcat (Tool for Operation Catalogues And Tables, http://www.star.bris.ac.uk/~mbt/topcat/ ), using the SAMP protocol (Simple Application Messaging Protocol, developed for the IVOA community http://www.ivoa.net/documents/REC/App/SAMP-20090421.html). We can use SAMP as a hub and route all messages between clients. It allows exporting and displaying 1D and 2D data products on these tools.
A simple and interactive connection between LATMOS SMDB and Topcat has been developed on LatHyS (the LATMOS web-interface).
LATMOS’ Webservices
2D or 1D data products which are not stored in the LATMOS SMDB, e.g. a 2D cut that is not included in the set of pre-computed archived 2D cuts, can be requested by IMPEx tools (e.g. CDPP-AMDA, CDPP-3DView, CLWeb or the IMPEx Portal) through web services, as defined in the IMPEx Protocol.
The eight released LATMOS web services are:
1. getFileURL: this web service returns URL/granules for 1D data product.
2. getDataPointValue: a generic method that can be used to return parameters for 0D (a given point), 1D (along a curve/trajectory), 2D (in a plane) or 3D (inside a volume).
3. getDataPointValueSpacecraft: this Method interpolates the physical simulation parameters along a given spacecraft.
4. getSurface: this method is called via CDPP-3DView. It is used to generate a meshgrid and compute interpolation for one or several parameters.
5. getFiledLine: this method is called via CDPP-3DView. It computes field or flow lines for requested footprints or passing through the S/C track.
6. getDataPointSpectra: This method returns ion spectra for a requested position or a list of positions.
7. getDataPointSpectraSpacecraft: this method returns ion spectra along the spacecraft track (the spacecraft is requested from the user).
8. isAlive: this method returns the status of the database (available or not).
Figure 20 shows a schematical description of one of the web services. A full documentation of LATMOS web services is provided online as xml documentation (see http://impex.latmos.ipsl.fr/Methods_LATMOS.html) and through the IMPEx Technical Documentation section that can alse be accessed via the IMPEx website (see http://impex-fp7.oeaw.ac.at/fileadmin/user_upload/pdf/ListofWebservices_for_LATMOS_v1.0.pdf).
Figure 20: Schema of input parameters required for one web service.
Outreach and Educational resources
The description of the model, the hybrid formalism and the archive is presented in the IMPEx Demonstrators webpage (see https://sites.google.com/site/impexfp7/) additional documentation can be found on the LatHyS website (http://impex.latmos.ipsl.fr/doc/Hybrid_model_documentation.pdf) and in D3.12.
Two tutorial videos on the interoperability of LatHyS and VO-Tools (Topcat, CDPP-AMDA and CDPP-3DView) are posted on the IMPEx Website (http://impex-fp7.oeaw.ac.at/84.html) and have been created in synergy with the Europlanet-RI project.
Paraboloid Magnetospheric Models (PMM)
The WG4 (SINP) main scientific task was to upgrade the generalized paraboloid model of the planetary magnetosphere, so that it could be used for the on the fly calculations in the IMPEx framework. The model is intended for description of the magnetospheric dynamics, taking into account the intrinsic planetary magnetic field, the magnetopause current magnetic field, the tail current system magnetic field, the magnetodisc magnetic field, the ring current magnetic field, and the interplanetary magnetic field, penetrated from the solar wind. Such modular structure allows us to represent the magnetic field inside the magnetospheres of Mercury, Earth, Jupiter, and Saturn.
Based on the Earth's magnetospheric space missions’ data, and the missions to Mercury (Messenger), Jupiter (Ullysses, Galileo), and Saturn (Cassini) systems, the important features and dynamics of these planetary magnetospheres were discovered. The Hubble Space Telescope (HST) images of the Jovian and Kronian aurora also brought a new finding related to the outer magnetospheres structure.
For the Earth's magnetosphere, the time dependent functions of the global magnetospheric parameters were found on the basis of the solar wind monitors’ data and geomagnetic indices. As a result, the current state of Earth's environment is represented as a function of solar wind parameters and geomagnetic activity level.
The Cassini measurements were compared with the modeled magnetic field in Saturn’s magnetosphere. The Kronian auroral UV images obtained by the Hubble Space Telescope, together with the Cassini solar wind magnetic field data gave us the dependence of Saturn’s outer magnetospheric and ionospheric dynamics on the solar wind dynamic pressure and the interplanetary magnetic field.
During the Messenger flybys Mercury’s dipole moment was estimated. A northward dipole offset about a half thousand km was found by minimization of differences between the model calculations and Messenger’s magnetometer data.
The scientific studies of the SINP group allowed the development of the model, which can be implemented in the IMPEx framework as the basis for real-time calculations of the magnetic field in the different magnetospheres.
The key objectives and milestones in the course of the project for the IMPEx group and in particular for the team at SINP were:
● Development and elaboration of the Paraboloid Magnetospheric Models for Earth, Mercury, Saturn and Jupiter.
● Scientific publications on the results of the PM Models development by SINP.
● Establishing a general approach to a unified access to the data producers (FMI, LATMOS, SINP, CDPP) by members of the IMPEx team.
● Support for SOAP web service technology as communication technique between the data producers and the IMPEx Portal, visualization tools etc.
● Support for NetCDF and VOTable.
● Defining main functionalities of data producers: value in points, along a line, on a surface, in a cube and magnetic field lines.
● Implementation of the web services defined for the planets: Earth, Mercury, and Saturn.
● Elaboration and improvement of the web services, together with web service consumers (IMPEx Portal, CDPP-3DView, CDPP-AMDA).
● Development of the SINP visualization tool prototype based on WebGL technology.
● Constant coordination with the IMPEx team during the project’s lifetime.
● Preparation of documentation for models and services.
Web services
Two approaches to the model usage have been developed via SINP web services: get – i.e. access to database and interpolation of pre-calculated data and calculate – i.e. direct calculations in real-time. Following the list of services, implementing these two principles:
getDataPointValue, getSurface, calculateDataPointValueFixedTime, calculateDataPointValue, calculateDataPointValueSpacecraft, calculateFieldLine, calculateCube, calculateDataPointValueMercury, calculateCubeMercury, calculateFieldLineMercury, calculateDataPointValueSaturn, calculateCubeSaturn, calculateFieldLineSaturn, calculateDataPointValueJupiter, calculateCubeJupiter, calculateFieldLineJupiter
Paraboloid Magnetospheric Model of the Earth
The magnetic field can be calculated in the solar magnetospheric coordinate system only inside Earth's magnetosphere, in the region -40 < x < 20 Re; -30 < y,z < 30 Re, which is bounded by magnetopause, represented by a paraboloid of revolution. Inside the magnetopause of the paraboloid model (PM) of the Earth's magnetosphere (Alexeev et al., 2003), the magnetospheric magnetic field of each large scale current system is determined by an analytical solution of the Laplace equation for the magnetic field scalar potential. The magnetic field component, normal to the magnetopause is assumed to be zero. The model represents the magnetic field inside the magnetosphere as a sum of the internal planetary magnetic field, given by the IGRF2015 model and the external one (Bm), represented by a superposition of the magnetic fields of the ring current, Br, the tail current system including the currents across a tail, and their closure currents on the magnetopause, Bt, the Region 1 field-aligned currents, Bfac the magnetopause currents screening the dipole field, Bsd, and the magnetopause currents screening the ring current magnetic field, Bsr:
Here BIMF is the interplanetary magnetic field partially penetrated into the magnetosphere. The model input values are the key parameters of the magnetospheric current systems, which represent their location and intensity:
● the geomagnetic dipole tilt angle Ψ,
● the magnetopause stand-off distance R1,
● the distance to the inner edge of the tail current sheet R2,
● the magnetic flux through the tail lobes Φ∞,
● the ring current magnetic field at the Earth's center br,.
The time dependent model parameters are calculated by the empirical data (solar wind density (n), velocity (v), Dst and AL indices, interplanetary magnetic field B-components (IMF_B) and by the current date/time using special submodels (Alexeev et al., 2003) optimizing parameter dependences on the specific sets of empirical data. Input model parameters could be specified by the user or taken fully or partially from the OMNI database for a given time moment (StartTime input parameter).
getDataPointValue
“get-” methods are based on the approach of interpolation values in the precalculated 3D-cubes in order to return values in the requested points. 3D-cubes are a set of three-dimensional arrays, which contain magnetic field vectors, calculated in points of a grid around planets. The tree.xml file contains a catalog of these pre-calculated 3D-cubes.
The web service getDataPointValue can be used to get a magnetic field vector, calculated by PMM in chosen points and inside a volume.
Magnetopause dimensions were determined by input model parameters of the corresponding run.
Figure 21: Components of the magnetic field vector calculated in points with getDataPointValue (Topcat).
calculateDataPointValueFixedTime
Calculations realized through “calculate-” services allow the model runs in real-time. The web service calculateDataPointValueFixedTime can be used to get a magnetic field calculated by PMM in chosen points in one time moment (i.e. a ‘static’ picture).
Figure 22: Magnetic field vector magnitude calculated in points with calculateDataPointValueFixedTime (Topcat).
calculateDataPointValue
The web service calculateDataPointValue can be used to get a magnetic field (calculated by PMM) in chosen points and in different time moments for each point (i.e. a ‘dynamic’ picture). Model parameters are retrieved from the database for each time moment (each point) - can be used for magnetic field calculation along a spacecraft trajectory that is provided by the user.
Figure 23: Magnetic field vector magnitude calculated in points with calculateDataPointValueFixedTime (Topcat).
calculateDataPointValueSpacecraft
The web service calculateDataPointValueSpacecraft can be used to get a magnetic field along the spacecraft trajectory in the chosen time period calculated by PMM. The spacecraft trajectory should be inside Earth's magnetosphere. Input model parameters are obtained from the database for each position of spacecraft and time moment within the chosen time period. Spacecraft trajectory is obtained through the CDPP-AMDA web service. Calculations can be made along the spacecraft's trajectories, provided by CDPP-AMDA.
Figure 24: l.t.r: Magnetic field calculated along Themis-A trajectory (CDPP-3DView). Example of the magnetic field visualization (Topcat). Magnetic field calculated along Themis-E and Cluster4 trajectories (CDPP-3DView).
calculateFieldLine
The web service calculateFieldLine can be used to get field lines (calculated by PMM), starting in chosen points and in different time moments for each point, including the magnetic field along these lines. Magnetic field lines starting points should be placed inside the valid region. Input model parameters are retrieved from the database for each field line. The user determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 25: Magnetic field lines, calculated from the given starting points with calculateFieldLine (Topcat).
calculateCube
The web service calculateCube returns a magnetic field, calculated by the Paraboloid Model in grid points of a cube. Time moment, cube boundaries and grid sampling can be set by user.
Figure 26: Visualization of the magnetic vector field magnitude, calculated with calculateCube, in the cube -20
getSurface
The web service getSurface can be used to generate a meshgrid and perform a magnetic field interpolation on the grid between values from a pre-calculated PMM 3D-cube. Input: a point and a normal vector to the requested plane as input parameters.
Paraboloid Magnetospheric Model of Mercury
The magnetic field can be calculated only inside the Hermean magnetosphere that is implied to be bounded by magnetopause, represented by paraboloid of revolution in the region -5.5 < x < 2.6 Rm; -4.1 < y,z < 4.1 Rm in the Hermean solar magnetospheric coordinates system. The paraboloid model (Alexeev at al., 2010) represents the magnetic field as a sum of the Hermean internal magnetic field RINT and magnetic fields of the tail current, Rt, the magnetopause currents, Rsd, and the interplanetary magnetic field partially penetrated into the magnetosphere of Mercury, RIMF:
Model input parameters (all are optional):
● BD - dipole field strength on the equator of Mercury,
● Flux - magnetic flux at the polar cap open field line region,
● RSS - subsolar magnetopause distance in Mercury radii (2439km),
● R2 - the distance to the inner edge of the tail current sheet,
● DZ - northern displacement of the dipole relative to the center of Mercury,
● IMF_B - components of the Interplanetary Magnetic Field penetrated into Mercury’s magnetosphere (in the HSM coordinate system).
calculateDataPointValueMercury
The web service calculateDataPointValueMercury can be used to get a magnetic field (calculated by PMM) in chosen points.
Figure 27: Components of the magnetic field vector calculated with calculateDataPointValueMercury (Topcat).
calculateCubeMercury
The web service calculateCubeMercury returns a magnetic field, calculated by Paraboloid Model, in grid points of a cube with chosen boundaries and sampling.
Figure 28: Components of the magnetic field vector calculated in the cube -7.5
CalculateFieldLineMercury
The web service calculateFieldLineMercury calculates magnetic field lines, which start in chosen points. The user determines the length of the field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 29: Magnetic field lines, calculated with calculateFieldLineMercury, along the spacecraft Messenger trajectory on the 2009/09/29 (CDPP-3DView).
Paraboloid Magnetospheric Model of Saturn
The magnetic field can be calculated only inside the Kronian magnetosphere, which is bounded by magnetopause, represented by paraboloid of revolution (in the region -1200 < x < 40 Rs ; -500 < y, z < 500 Rs in the Kronian Solar-Magnetospheric coordinate system). The paraboloid model (Alexeev at al., 2006) represents the magnetic field as a sum of the Kronian internal magnetic field BINT (Burton et al.,2010) and magnetic field of the external sources Bm, which is a superposition of the magnetic fields of the magnetodisc, Br, the tail current system including currents across the tail and their closure currents on the magnetopause, Bt, the magnetopause currents, Bsd, and the interplanetary magnetic field partially penetrated into the Kronian magnetosphere, BIMF:
Model input parameters (all are optional):
● BDC - magnetic field at the magnetodisc (MD) outer edge
● bt - minus Z-component of the magnetic field at the tail current sheet inner edge
● RD2 - distance to the inner edge of the magnetodisc,
● RD1 - distance to the outer edge of the magnetodisc,
● R2 - distance to the inner edge of the tail current sheet,
● Rss - magnetopause stand-off distance,
● IMF_B - components of the Interplanetary Magnetic Field penetrated into the magnetosphere in the KSM coordinate system.
For each input model parameter a user can set a value manually or take a value by default.
calculateDataPointValueSaturn
The web service calculateDataPointValueSaturn returns a magnetic field (calculated by PMM) in chosen points with different timestamps but one set of parameters for all the points.
Figure 30: Magnetic vector field magnitude calculated with calculateDataPointValueSaturn (Topcat).
calculateCubeSaturn
The web service calculateCubeSaturn returns a magnetic field, calculated by the Paraboloid Model, in grid points of a cube. Cube boundaries and grid sampling can be set by user.
Figure 31: Components of magnetic vector field in the cube -20
calculateFieldLineSaturn
The web service calculateFieldLineSaturn calculates magnetic field lines that start in points chosen by the user who also determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 32: Magnetic field lines by calculateFieldLineSaturn, along Cassini trajectory (CDPP-3DView).
Paraboloid Magnetospheric Model of Jupiter
The magnetic field can be calculated only inside the Jovian magnetosphere, which is bounded by magnetopause, represented by paraboloid of revolution (in the region -450 < x < 150 Rj; -300 < y, z < 300 Rj in the Jovian Solar-Magnetospheric coordinates system). The paraboloid model (Alexeev and Belenkaya, 2005) represents magnetic field as a sum of the Jovian dipole magnetic field BINT (System III model) and the magnetic field of the external sources Bm, which is a superposition of the magnetic fields of the magnetodisc Br, the tail current system including tail currents and the closure currents on the magnetopause Bt, the magnetopause currents Bsd, and the interplanetary magnetic field partially penetrated into the Jovian magnetosphere BIMF:
Model input parameters:
● BDC - magnetic field at the magnetodisc (MD) outer edge,
● bt - minus Z-component of the magnetic field at the tail current sheet inner edge,
● RD2 - distance to the inner edge of the magnetodisc,
● RD1 - distance to the outer edge of the magnetodisc,
● R2 - distance to the inner edge of the tail current sheet,
● Rss - magnetopause stand-off distance,
● IMF_B - components of the Interplanetary Magnetic Field partially penetrated into the Jovian magnetosphere in the JSM coordinate system.
For each input model parameter a user can set a value manually or take a value by default.
calculateDataPointValueJupiter
The web service calculateDataPointValueJupiter returns a magnetic field (calculated by PMM) in chosen points with different timestamps, but one set of parameters for all the points.
Figure 33: Magnetic field vectors calculated in points of the Galileo spacecraft trajectory on the 2002/11/01 with calculateDataPointValueJupiter (CDPP-3DView).
calculateCubeJupiter
The web service calculateCubeJupiter returns a magnetic field, calculated by the Paraboloid Model, in grid points of a cube. Cube boundaries and grid sampling can be set by the user.
Figure 34: Components and slices of magnetic vector field in the cube -100
calculateFieldLineJupiter
The web service calculateFieldLineJupiter calculates magnetic field lines that start in points chosen by the user. The user determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 35: Magnetic field lines, calculated with calculateFieldLineJupiter, along Galileo trajectory on the 2002/11/01 (CDPP-3DView).
The IMPEx Portal
The IMPEx Portal has been fully developed at the IWF in Graz in the course of the bachelor and master thesis of Florian Topf (also see chapter 1.4.2). The IMPEx Portal has been defined as part of WP5 in RP2, in accordance with the Project Officer Antonio Fernandez-Ranada Shaw, since one of its main purposes is to attract new users to IMPEx and to introduce the capabilities and hence the potential of IMPEx in a user friendly and comprehensive way.
The development phase of the IMPEx Portal was concluded successfully in RP4. All major features derived from various requirements obtained e.g. by the advisory boards have been implemented and validated. The portal is now a central element of the IMPEx environment and a one-stop-solution for any interested parties and potential users. It allows obtaining a quick, practical insight into available tools as well as methods on a lower level of the system. The portal map (see Figure 36) connects all IMPEx enabled tools and also allows browsing, querying and transferring data from one of the attached SMDBs to e.g. CDPP-AMDA via SAMP. All queries are conducted in a unified internal registry that is updated on a daily basis, pulling in meta data from all attached SMDBs. It also conducts an “is alive” test at start-up and hence is an effective monitoring tool of the IMPEx environment including all attached components of the system.
Access to the IMPEx Portal is completely free and no user credentials must be provided for access. Nevertheless users can store selections made and results obtained via methods calls in the local browser storage for later access. This way data can be saved for later use, without the need of authentication. The portal is developed using HTML5/CSS3 technologies and can be operated on desktop computers as well as mobile devices without restrictions. To support the latter it can adapt to the actual screen size, offering an optimal experience for on all devices
Figure 36: The IMPEx Portal showing tab of the "portal map".
As can be seen in Figure 36, the portal offers several different sections via a tapped interface, these are briefly described in the following:
• Section “Map”: The portal map is the main interface shown per default, detailing all IMPEx enabled tools and SMDBs and allowing to query data, call methods, store results, transfer data etc.
• Section “Config”: Allows accessing the IMPEx configuration file as xml as well as json. The configuration is also parsed and displayed in a human readable form.
• Section “API”: This section offers a REST based interface to the IMPEx Protocol. Further it offers methods to retrieve the configuration file and query the internal (meta-)data registry.
• Section “Support”: This section offers a simple way of contacting the IMPEx team with regard to specific (technical) questions. At least one expert of each team is part of the list of recipients - the form offers select boxes to specify to which tool or part of IMPEx the question is related to (consequently also indicated in the mail’s subject line). All questions posed will be listed, thus building up a (searchable) FAQ in the future.
• Section “User Manual”: A comprehensive manual as a PDF for all types of users of the IMPEx Portal. The manual also features examples and step by step descriptions, including numerous screen shots of real use cases. The document is available at:
https://sites.google.com/site/impexfp7/home/impex-portal-user-manual
• Section “Tool Docs”: This section provides the QuickGuide to IMPEx Tools, which was motivated by feedback obtained from the advisory boards in RP3. The QuickGuide gives a brief overview of IMPEx and its available tools, and thus directs the user to the tool that best fits his/her needs or use case. The document is available at:
http://impex-fp7.oeaw.ac.at/fileadmin/user_upload/pdf/IMPExToolsandtheirUsability.pdf
Potential Impact:
This chapter details the potential impact of the IMPEx project on planetary science in general and R&D activities in this field, including the socio-economic and the wider societal implications of the project so far as well as the main dissemination activities and exploitation of results.
Impact of technological developments of IMPEx
One of the main achievements of IMPEx-FP7, the definition of a comprehensive data model applicable on numerical simulations in the area of space plasma physics and the plasma environments of planetary objects, is expected to have a significant impact on planetary science. Until the advent of the SPASE simulation extensions (which have been directly derived from the IMPEx Data Model, also see Figure 37 and D3.14) numerical models mostly used proprietary data structures and metadata descriptions to store and process data, obtained via modelling runs. This of course considerably increased the complexity for modelling and research groups to exchange data and solve scientific and technological tasks in a concerted effort, involving data from multiple sources. Since data needs to be consistent and described in a homogenous way, each team more or less had to translate back and forth between the data structures used by the different group respectively – this is of course not necessary any more, as soon as all teams agree to use one common data model. However, this again is only feasible, once a format exists that satisfies the requirements of all parties involved and allows to describe all relevant aspects of the data in relation to the respective scientific investigation at hand.
Figure 37: A diagram of the IMPEx Data Model - the first complete data model of its kind that has been finalized, documented and fully released for public use, allowing to fully describe modelled space plasma data.
The IMPEx team is convinced that the SPASE simulation extensions fulfil the basic requirements of a wide range of research carried out in space plasma physics. Moreover, since it is a living definition, the model can be extended in the future to include further elements that will enable new users to describe aspects required for their specific research. To this behalf, processes have been defined within the SPASE group, and this way new additions to the data model are regularly discussed in web conferences and added to the definition, once agreed by all parties and stakeholders involved. Hence, it is expected that this new data model based on SPASE (already widely used in the field of observational data in the field of space plasma physics) will foster the exchange of data between different modelling groups and even more so open up data archives formerly only used and even accessible within specialized mission teams to the wider scientific community.
Since communication among experts and scientists is of course crucial in research, the impact of a common data standard, once adopted by a sufficiently big number of researchers, archives and institutions cannot be underestimated. The less time is lost in administrating and distributing the data, and the easier it is to compare results obtained by different groups, the more efficient research can be carried out. This is of course achieved by enabling automated tools and applications to handle data in a consistent and transparent way, thus freeing users from having to deal with the specifics of different formats. The more tools are enabled to access and process data, the more users are reached and the bigger the impact hence is. To further facilitate the interoperability of tools, IMPEx created a common protocol that covers a broad range of requirements. In IMPEx this was achieved primarily by defining a set of web services (i.e. the IMPEx Protocol) that allows different tools as e.g. CDPP-AMDA, CDPP-3DView and various SMDBs to exchange data and call functionalities taking these data, stored compliant with the IMPEx Data Model (i.e. SPASE simulation extensions which is an inherent component of the protocol), as input. This way all users of this set of IMPEx enabled tools can access a common data pool and researchers, students, post docs and the public in general have access to all information and functionalities of the IMPEx environment.
As with the data model, the set of web services can be extended as well, to cover requirements in the future that were not thought of or even viable during the project’s inception or lifetime. Moreover, the definition can also be taken as a basis for proprietary extensions by other institutions, while preserving backward compatibility, taking advantage of the fact that many problems one is faced with when designing a coherent set of web services to handle plasma data, have already been successfully solved in IMPEx. It should also be noted that the IMPEx Protocol is also able to handle models for real-time calculations, as well as pre-calculated cube data stored in an SMDB (see Chapter 1.3.3 for further details on real-time methods).
Since SOAP is used for the basic implementation, tools and web apps all over the world can harvest the available functionalities more or less automatically and bring them into their respective software landscape, hence making them accessible and workable for their specific community. This way the idea of IMPEx will also spread across the web and it can be expected that it will encourage other projects to follow a similar path or to seek cooperation with institutions that are part of the original project team - see D5.3 for more information on projects and institutions that are already collaborating with IMPEx. If users run into access restrictions at certain tools or SMDBs connected to IMPEx, they can simply contact the respective institution providing the service and ask for a free account. In any case the philosophy behind IMPEx is to offer as much data and functionalities for free as possible, which is an approach that is highly beneficial for scientific endeavours in general. The more tools and applications in the scientific world are able to interoperate without restriction, the easier it is to distribute information and collectively tackle problems that are difficult to solve for a single entity, researcher or institution. Following this approach has been shown to be very successful in the context of IMPEx, and it is to be expected that this advantage scales up very well to the scientific community as a whole. After all, as already briefly discussed in Chapter 1.2 there are also other initiatives, as e.g. IVOA that are following this approach with great success.
Information technology naturally is an essential part of modern science, and building solutions that are designed to provide certain minimum standards with regard to usability and stability is a time consuming task that requires professional software developers to be engaged for considerable time spans. It can’t be stressed enough that a lot of resources are freed up and thus made available for pursuing the actual research, when (IT related) solutions already implemented and validated at one institution can be re-used at other institutes and projects, either solving the task altogether or providing a stable basis on which further, more specialized solutions can be built upon. In a nutshell, IMPEx thrives to provide the common language that allows this close interaction to take place on a technical level as well.
With regard to access restrictions, it should also be stressed that all functionalities available at the IMPEx Portal, i.e. all methods offered by participating nodes (tools and SMDBs) are fully accessible without any credentials required at all, and hence free of charge. The same is true for most of the data that can be browsed and downloaded via the portal interface, or the respective SMDB providing the data in the first place. However, there are of course (observational) data sets, e.g. offered by the tool CDPP-AMDA that are not yet fully released by the respective PI, and thus require users to log-on to the system with specific credentials, in order to be able to access them. However, this is completely outside of the influence of the project members of course.
Socio-economic and wider societal implications
Software development is a costly endeavour since applications and software environments satisfying up-to-date standards with regard to usability and stability require considerable effort in terms of design, development as well as testing and validation. Thus, purely scientific institutions rarely have the time and resources available to develop customized software in a coordinated way, which leads to a scattered software landscape in the field of science tools that often cannot satisfy the true needs of its user communities. Here projects like IMPEx can make a valuable contribution, since dedicated funds are available and professional software development can be conducted via e.g. sub-contracting of development companies as e.g. GFI in IMPEx engaged by CNRS for the development of the visualization software CDPP-3DView. Apart from the implementation of ready to use applications, also the definition end extension of standards is a vital part of modern software development, keeping in mind that without standardization and the definition of appropriate interfaces no true interoperability of different applications can be achieved. As with writing code, the definition of sound, stable and comprehensive standards and interfaces requires a considerable coordinated effort, involving all or as many stakeholders as possible, in order to produce a meaningful definition that will actually be leveraged in future development efforts. IMPEx has not only produced ready to use software, but also put a lot of thinking, time and effort into the framework itself, i.e. the IMPEx Data Model and the IMPEx Protocol (including further definitions as e.g. the IMPEx Configuration etc.).
This broadens the base of stable implementations and definitions available in order to build specialized, heavy tailored solutions on top. The latter can then be again performed much more easily by institutions that have a strict scientific focus. Here of course the flexible architecture of IMPEx, relying on a standard data model and a set of web services that have been designed with the principle of generality in mind, provides a valuable contribution to the (planetary) science community. More efficient software tools naturally lead to better scientific results and more productiveness by minimizing administrative and purely technical tasks, allowing scientists to focus on the actual investigation and scientific questions at hand.
Another important aspect are the numerous tutorials (video, speech and textual), and the technical documentation produced during the project. The videos can be used as a quick and comfortable way of getting an overview of the possibilities offered by IMPEx, while the documents (as e.g. the Hybrid Model Demonstrators or the IMPEx Tutorial for CDPP-AMDA & CDPP-3DView) provide all required information and resources to then deepen the knowledge in specific (scientific and technical) areas, or with regard to concrete tool support needed. The IMPEx Portal which is designed to be a one-stop-solution for newcomers to IMPEx is also covered by a dedicated and comprehensive tutorial that allows potential users to quickly get started. These materials can also be used by students and senior classes as well as teachers and at university level in general, in order to prepare classes and gather material for talks and presentations. It is the hope that the educational materials and tools provided will help to encourage pupils and students to pursue a career in space physics. The visual tools as e.g. CDPP-3DView are particularly suitable in order to attract attention and generate interest through presentations etc. All tutorials can be accessed on the IMPEx website at the IMPEx Demonstrators section at https://sites.google.com/site/impexfp7/ - IMPEx tutorial videos can be viewed directly at the IMPEx Videos section, as well as from the HYB Code channel available on YouTube – the latter featuring videos for FMI’s hybrid and MHD codes, explaining their theoretical background as well as their usage via IMPEx and the SMDB interface provided by FMI.
An important socio economic impact of IMPEx is the fact that many positions at the participating scientific institutions were created, and in particular post-docs and students were able to work for the project and gain valuable (first) experiences and insights into new technologies and IT related approaches.
Figure 38: The IMPEx Team at PMC 36 in Moscow (May 2014). Florian Topf (2nd from right, bottom row), Manuel Scherf (3rd from right, middle row) and Tarek Al-Ubaidi (first from right, top row).
At IWF also a project management position was supported during project execution, held by Tarek Al-Ubaidi, who brought many years of experience in the field of professional software development, web development and knowledge management into the project. Also at IWF, Florian Topf was engaged for development work – in the course of his employment Florian Topf (supported by Tarek Al-Ubaidi) worked out a comprehensive concept for the IMPEx Portal, and successfully finished his bachelors and then also his master thesis focusing on the IMPEx portal, scientific tools in general, and the benefits of functional programming in this field. Tarek Al-Ubaidi also served as a supervisor for the master thesis of Florian Topf. Finally Florian Topf also implemented the portal according to the design agreed by the team at a face-to-face meeting at the end of RP3 (in the course of PMC #36 in Moscow), using scala, an innovative run time environment and development platform, based on the functional programming paradigm. Also at IWF Manuel Scherf was engaged as a research assistant, supporting the team in the creation of tutorials and the editing of scientific material from a didactic point of view. Further Manuel Scherf was pivotal in validating software products that were created or extended in the course of the project, as e.g. the visualization software CDPP-3DView or the IMPEx Portal in general.
At SINP (Moscow State University) Lucy Mukhametdinova was engaged, starting at the beginning of RP3. She was responsible for supporting Vladimir Kalegaev in implementing, testing, bug fixing and last but not least documenting the extensive web service interface provided for the paraboloid model and the SMDB at SINP.
At FMI David Perez-Suarez did extensive experimentation with regard to workflow integration of the IMPEx toolset, in particular in connection with the scientific workflow system Taverna (also see periodic reports for RP2 and RP3) that was used to build prototype workflow support for IMPEx on the basis of the IMPEx Protocol.
AT CNRS/LATMOS Sebastién Hess was engaged as a post-doc in RP2 and RP3, in order to support the definition process of the IMPEx Data Model. Sebastién Hess subsequently took the lead in this process and made a decisive contribution to this effort that resulted in the successful finalization of the prototype version of the data model in RP3.
As already discussed in the course of technical impacts, IMPEx, through its very nature of being a software environment, providing the means for tools and services to communicate and exchange data, very well supports and even fosters the cooperation of different institutes, not only on a purely technical, but a scientific and educational level as well. Hence many new cooperative links have been either newly established in the course of the project, or deepened due to a now available (technical) infrastructure that allows to do things that were either impossible or of a too high effort prior to IMPEx. At the end of RP4 IMPEx has active links to the following projects and infrastructures (also see D5.3 for further details):
● SPASE (http://www.spase-group.org/)
● Europlanet/IDIS (http://www.europlanet-idis.fi/)
● HELIO (http://www.helio-vo.eu/)
● CCMC (http://ccmc.gsfc.nasa.gov/)
● ETFLA/Astronet (http://www.astronet-eu.org/)
● ASTRONET (http://www.astronet-eu.org/)
● VESPA (http://vespa.obspm.fr/)
The level and nature of cooperation is of course dependant on available funding, as well as the focus of the respective institution involved. While e.g. cooperation with ASTRONET is primarily of a scientific nature, the ties to the SPASE community are more or less purely technical. Links to CCMC and Europlanet were primarily driven by WP2, see chapter 1.3.1 for further details.
Further cooperative ties are built, as this report is being written. One of them concerns a technical as well as scientific link to UCLA; here the integration of simulation results is being prototyped and the respective data is already part of the IMPEx Configuration. Another link is being established to LESIA (http://www.lesia.obspm.fr).
The impact of these scientific and technological ties is often considerable, given that developments at one institution often drive or spin-off further developments at other institutions that would not have been possible or feasible otherwise without the groundwork being laid. By taking advantage of synergies, it is often possible to minimize the effort involved in following a certain research topic, or develop a solution for a highly specialized problem, specific to a given scientific use case.
Dissemination activities and exploitation of results
IMPEx has been presented at numerous scientific conventions and congresses, including the annual EPSC (European Planetary Science Congress, organized by the Europlanet community), the EGU (European Geosciences Union) and other important events, covering the field of planetary science. These events included poster presentations as well as oral presentations, hands on sessions and talks on various subjects of the IMPEx projects, ranging from technological to purely scientific topics.
In June 2012 Tarek Al-Ubaidi e.g. joined the Planetary Data Workshop for Users and Developers that took place in Flagstaff Arizona. The oral presentation that was received with great interest also resulted in a closer exchange with Dan Crichton (PDS Engineering Node Manager at NASA/JPL), with regard to efforts related to the (then on-going) definition process of the xml based PDS4 data model, to integrate modelled plasma data into PDS4, as the IMPEx simulation extensions are doing in the context of SPASE. Dan Crichton consequently also joined the IMPEx User Support Board in 2012 – please refer to Table 2 for a complete list of dissemination activities for the entire project duration (until June 2015). It is expected that IMPEx and its technological as well as scientific results will continue to be referenced extensively in publications and talks, in particular those held by members of the IMPEx core team (IWF, FMI, CNRS, and SINP). As an example this year’s EPSC taking place in October in Nantes, France, will include a poster presentation on possible follow up actions and projects, talking the ideas of IMPEx to the next level, also integrating cloud resources and advanced data discovery technologies.
Starting in RP2, also annual briefing sessions were held to inform the board members (USB and SSB boards, see D1.3 for details) of all the latest developments, and to gather valuable feedback that was consequently processed and henceforth strongly influenced further developments as well as the planning process. The live events held as face-to-face meetings at EGU in Vienna in RP2 (2013) and RP4 (2015), were public in the sense that any participant of EGU could take part.
Several press releases detailing the project goals and current status were written and subsequently published throughout the project. As captured in D5.2 of WP5, two articles were written by the IMPEx team (under the lead of IWF-OeAW) and published in the Parliament Magazine in RP1 and RP2 respectively. In the latter half of the project, the Vienna based PR company PR&D, specialized on the publication of scientific content, wrote an article in close cooperation with IWF-OeAW and with input and feedback from the entire IMPEx team. The publishing effort by PR&D turned out to be very effective; the first article in RP3 (titled Ready For Take-Off - Launch of New Data Model Boosts Space Science) reached over 60 confirmed coverages. The second article by PR&D published in RP4 (European Team Creates Universal "Language" for Space Science) reached 74 confirmed coverages (May 2015). The media outlets also included local newspapers (e.g. Kleine Zeitung) technical journals (e.g. Computerwelt or Deutsches Ingenieurblatt) and scientific publications (e.g. science 2.0). The articles by PR&D were available (and published) in German as well as English language. In RP4 also Finnish and Russian versions were prepared and published locally. The team at IWF-OeAW also prepared a publicly accessible directory that included several images that could be published along with the text provided. Since RP3 there is also an IMPEx Press Kit available from the website that can be used by journalists and other interested parties to gain further background knowledge on the project, its content and goals.
The IMPEx Website (also see chapter 1.5) also includes a video section that currently features six video tutorials on the following subject:
• IMPEx Tutorial Video - Tutorial demonstrating the combined use of the IMPEx Portal and several IMPEx enabled tools (IWF-OeAW, ~10 min.).
• Tutorial on interoperability of CDPP-AMDA, LatHyS and Topcat - this video was created by Europlanet in cooperation with IMPEx using funding from FP7/REA, and presented at EPSC 2013 (~7:30 min.).
• Tutorial on CDPP-AMDA, CDPP-3DView and Simulation Databases (SMDBs) - this video was created by Europlanet in cooperation with IMPEx using funding from FP7/REA, and presented at EPSC 2013 (~10:30 min.).
• Auroral Processes of Saturn - This video was created by Europlanet with funding from FP7/REA and presented at EPSC 2013 (~9:00 min.).
• Tutorial on IMPEx Matlab Tools (~9:40 min.)
• Tutorial on Hybrid Web Archive (~06:40 min.).
Further there are nine videos (mostly in Finnish language) available from the following HYB video channel, provided and maintained by FMI: https://www.youtube.com/channel/UC-auXDoJSbYBVrSsI2-NpPg The channel deals on various subjects surrounding the hybrid code (HYB) as provided by FMI, see Table 2 for further details.
The IMPEx Podcast has been part of the project from the start and issued 16 episodes in total over the whole project duration of four years. Each episode featured a guest from the wider (planetary) science community covering a topic that is related to the project. Many of the episodes featured members of the IMPEx core team, including the coordinator, deputy coordinator, scientific representative, work package and task leaders as well as members of the advisory boards. The podcast is an ideal way of becoming acquainted with the project, its goals and main achievements. All episodes are hosted and produced by Tarek Al-Ubaidi and can be listened to via the IMPEx website: http://impex-fp7.oeaw.ac.at/podcast.html. Also see Table 2 for a complete list of episodes including titles and featured guests.
The IMPEx Portal that has been implemented in the course of WP5 (dedicated to outreach and dissemination) is also an important instrument for attracting new users to the system. The portal can be accessed with any desktop or mobile device, capable of displaying and executing HTML5/JavaScript based web apps, and collects all relevant information at one spot. Users can enter the portal without having to provide any credentials and are free to browse data provided by SMDBs or to call methods of the IMPEx Protocol, supported by a graphical and easy to use interface. It is an attractive way of making oneself familiar with all relevant IMPEx capabilities (including the IMPEx Configuration, the REST based web service interface etc.) and hence become involved on a very practical level. This way the sometimes rather abstract definitions and concepts of IMPEx (in particular for non-IT savvy users) become much clearer, easy to grasp and fully leverage. Please refer to Chapter 1.3.4 for further (technical) details on the IMPEx Portal.
List of Websites:
The IMPEx website (also see Figure 39) is the central access point for all IMPEx relevant information. It provides access to comprehensive technical documentation, tutorials, videos and the IMPEx Podcast. Further there is general information about the team, links to publications & talks, articles and news on latest developments of the project.
The IMPEx website can be reached via: http://impex-fp7.oeaw.ac.at/
The website is administrated and moderated by Tarek Al-Ubaidi (tarek.al-ubaidi@oeaw.ac.at, +43 316 4120 673) and Manuel Scherf (manuel.scherf@oeaw.ac.at, +43 (316) 4120-672). For technical problems please contact Manfred Stachel (Manfred.stachel@oeaw.ac.at, +43 (316) 4120-412).
IMPEx was a four year collaborative project, funded under the 7th framework of the European Union (FP7) that started in June 2011. Its consortium is made up of four members, scientific institutions from Austria (IWF-OeAW, the coordinating entity), Finland (FMI), France (CNRS), and Russia (SINP). The projects main goal is the creation of a software environment that facilitates and supports the comparison and overlay of modelled space plasma data vis-à-vis real world observational data, obtained in the course of past and future space missions. The software environment is consisting of several well established tools (mainly CDPP-AMDA, CDPP-3DView and CLWeb) provided by CNRS, for the analysis and visualization of space mission data. These tools have been extended to handle various aspects of simulation data with the goal of gaining a better understanding of observed phenomena, as well as to optimize existing simulation codes and related models. IMPEx enabled tools are aimed at supporting mission planners and spacecraft designers, by providing modelled and empirical data for e.g. virtual space flights in simulated planetary (plasma) environments.
During the first project year the concepts and goals outlined in the proposal were concretized in the course of the requirements definition (supported by the creation and analysis of scientific use cases) and architectural design phase (see D2.2 and D2.3). Several approaches were discussed and finally agreed upon and carefully documented. The main pillars of the IMPEx system were created; the IMPEx Protocol and the IMPEx Data Model basically cover all requirements from a technical and conceptual perspective, and allow for the exchange of data between various nodes of the topology as captured in the IMPEx Configuration. In the second and third year of the project the focus was primarily set on the finalization of all designs and of course the implementation, including the preparation of test cases for testing and system validation. In this period the project’s advisory boards were an important source of feedback and orientation for the team, with the aim to provide a system that is relevant and capable of solving real life problems in the course of scientific investigations. The boards are also pivotal in capturing all relevant requirements for the IMPEx Portal that was developed at IWF-OeAW during the second half of the project. It provides a one-stop-solution for anyone who is interested in IMPEx and wants to learn about the system through a hands-on experience, covering all aspects of IMPEx and integrating all tools of the environment.
The final project year was dedicated to the finalization of remaining definitions, features and functionalities deemed essential by the user community, as well as an integrated test effort to assure the required stability and general quality of the system. Further the IMPEx Data Model became an official part of the SPASE data model, upon which it is based on. The IMPEx Data Model hence becomes the SPASE simulation extension that will enable all users and applications of SPASE to integrate simulation data into their (database) systems, being able to handle and search simulation data as it is also possible for observational data.
The IMPEx website was established very early on during the project and provides comprehensive access to all relevant information (technically and scientifically) in order to gain a detailed insight into IMPEx, its tools and capabilities. The website offers a rich selection of video tutorials, comprehensive online material as well as a dedicated project podcast, featuring team members, members of the advisory boards etc. in 16 episodes, covering a wide range of subjects about and surrounding IMPEx. All in all it can be said that IMPEx exceeded expectations by its protagonists, in particular by setting a de facto standard in the field of simulation data and by establishing a prototype system that is based on common design principles of the Virtual Observatory (VO) and planetary science community.
Project Context and Objectives:
Europe as a leading participant in international endeavours (e.g. Rosetta, Mars Express, Venus Express, ESA/NASA Cassini-Huygens) to explore our solar system and planetary objects is also at the forefront in the development of theoretical models to simulate conditions in space. Empirical data derived via actual measurements during space missions and data obtained through computational model runs are two major aspects of modern space research. While it is obvious that models can only be further developed by comparing the results to real word measurements, it is as important to then use these models (once sufficiently confirmed by observational data) to interpret new measurements and to understand the physical processes behind the data. Both processes need adequate software support that was practically not available in planetary science by the time the project idea of IMPEx was developed. Once realized, modelled data can also be easily applied to support spacecraft designers and engineering in general, by providing the means to simulate conditions in planetary environments to e.g. obtain accurate requirements for spacecraft hardening, and design requirements in general. The last major application is to fill measurement gaps in observational data using modelled data derived via validated (sic) models.
Figure 1: The IMPEx environment with its main tools and SMDBs, connected through the IMPEx Protocol that is fully implemented by the IMPEx Portal. All data is stored compliant with the IMPEx Data Model and can be easily exchanged between the various nodes (tools and SMDBs).
The consortium of IMPEx includes several institutions that have developed world renowned computational models in the field of space plasma physics, being able to simulate the (plasma) environments of the majority of planetary objects in the solar system, including moons and comets. The IMPEx modelling sector is comprised of two hybrid models provided by FMI and LATMOS (CNRS), a magneto hydrodynamic model by FMI, as well as a unique paraboloid model provided by SINP – the models in detail are:
- The worldwide recognized 3D hybrid modelling platform HYB for study of Solar System objects’ plasma environments (developed and hosted at FMI)
- Global MHD modelling platform for 3D terrestrial magnetosphere (developed and hosted at FMI)
- The global 3D Paraboloid Magnetospheric Model for simulation of magnetospheres of different Solar System objects (developed and hosted at SINP). This model also offers on-demand calculations (see IMPEx Protocol), i.e. responses are more or less real-time.
- The LATMOS hybrid model developed and hosted at Université de Versailles Saint-Quentin.
Further, several web based analysis tools (CDPP-AMDA, CLWeb, the Java based visualization tool CDPP-3DView, see Figure 2) and user interfaces for the simulation databases (SMDBs) provided by FMI, SINP and LATMOS are part of the environment. The user interfaces of FMI and SINP also offer WebGL based visualization capabilities (Figure 1).
Figure 2: Visualization of a magnetic field simulation run interpolated onto the orbit of Mars Express, visualized via the Java based 3D tool CDPP-3DView provided by CDPP. The tool has been extended with regard to visualization capabilities required for the visualization of magnetic field and plasma data and enabled to communicate with the IMPEx environment (IMPEx enabled tool).
One of the main technical objectives of the project was to enable these tools to exchange data with all participating SMDBs and hence integrate data obtained through complex simulation runs with observational data that is already being processed and analysed by the aforementioned tools (the available data was also significantly increased in the course of IMPEx, see D2.13). Another (scientific) objective is to further develop models in particular the paraboloid models for Mercury, Jupiter, Saturn and Earth by SINP (also see D4.4 D4.5 and D4.6).
Connecting tools and databases is a challenging task, in particular when complex scientific data is involved. Numerous problems must be tackled as e.g. the aspect of generality, i.e. the ability to scale and reuse the solutions defined and implemented in other systems, and to extend the “data network” far beyond the original set of software applications and data collections.
In this sense the approach of IMPEx is highly compatible with the vision of the International Virtual Observatory Alliance (IVOA) that calls for astronomical datasets and other resources to work as a seamless whole. The philosophy followed here is that it is unproductive to put effort in “reinventing the wheel”, i.e. to implement similar functionalities over and over again in different tools. Instead the IVOA initiative, as does IMPEx, aims at connecting systems in order to be able to easily exchange (scientific) data and leverage functionalities of a rich set of tools. Thus, IMPEx uses several standards that originated out of the VO community that were defined by IVOA. Among them is the VOTable format that is heavily used in IMPEx, to exchange trajectories etc. SAMP is another IVOA standard used that allows web based tools to communicate with each other in a straight forward way, circumventing the back ends.
Besides scientific and technological aspects, there is also a strong relation to public education as well as a significant socio economic component in general. IMPEx delivered a wide range of materials that can be used in public education at the level of public schools as well as higher education including university level courses (see D2.14 D2.15 D3.12 D4.7 and D5.2). The website, which has been a central element for public outreach as well as an essential hub for technical information, features a number of tutorials, videos and extensive documentation of tools, models and the scientific approaches that lie underneath the environment. It is the hope that these materials will help to encourage young students to seek careers in space science and to engage in exciting research that is conducted in Europe and all over the world. In addition to that, the IMPEx Data Model (see Figure 37) provides a pivotal prerequisite in order to foster the exchange of scientific data, derived from computational models.
The IMPEx Data Model is, and will be even more so in the future, helping to bring the data and models outside of mission teams and specialized modelling groups, making them accessible and useful for the broad planetary science community, and, in this way, promoting the contribution of space assets to scientific and technological knowledge and development. Existing levels of scientific data exploitation, cooperation, as well as technical capabilities for communication of researchers including access to remote databases and computing infrastructures, provide ideal conditions for the integration of a combined modelling and data environment, as IMPEx is representing today.
As already briefly mentioned in Chapter 1.1 the IMPEx Data Model became the SPASE simulation extension and hence an official part of SPASE, which is one of the leading and most widely applied data models in space plasma physics worldwide. By integrating the IMPEx definitions, SPASE now is capable to also reflect on plasma data that was derived by computational modelling, and hence becomes the ideal environment for a combined use of both types of data. Since the IMPEx simulation extensions are conceptually close to the corresponding data structure and definitions for empirical plasma data, it is now a straight forward task to match data sets from both worlds. To use a practical example, as we can see in Figure 37, NumericalOutput is related to a certain Repository just as is NumericalData used in “traditional” SPASE. However, in the context of the simulation extensions the ontology is adapted, and the Instrument element used for empirical data becomes the SimulationModel, to clearly specify the technical source of the data so it can be adequately assessed.
With the development of the IMPEx Protocol a commonly used approach in IT development has been followed, and a set of SOAP web services was defined, implemented and offered (via *.wsdl files) to the worldwide community, in order to enable it to fully leverage functionalities of the IMPEx environment. The idea is that through the provision of a common mechanism to expose modelled data including functionalities (i.e. web services) that can be applied on the data, also other SMDBs and infrastructures worldwide are encouraged to join the community and integrate their data, ever increasing the scientific scope of IMPEx. In mid-2015 there are already advanced prototype implementations available, and data from UCLA has e.g. been added to the IMPEx Configuration. There are many other projects (besides SPASE) and initiatives that benefit from IMPEx developments and vice versa that are cooperating with the project as e.g. EFTLA/Astronet, CCMC, VESPA - see D5.3 for further information.
New requirements coming up in the future can be addressed straight forward through further updates of the IMPEx Protocol by either adding new methods, extending existing methods (preserving backward compatibility) or by extending the IMPEx Data Model to be able to describe an even broader range of simulation data and models. This way IMPEx is to be regarded as a living definition that will grow along with its applications and future requirements.
New horizons could also be reached by integrating cloud computing capabilities in possible follow up projects. Already within IMPEx FMI has done some experimentation with cloud technologies, by deploying the HYB modelling code on cloud resources, in order to test integration of these technologies in possible follow-up projects. Since modelling runs are a vital part of IMPEx, the inclusion of cloud resources would also enable smaller institutions to do complex runs on-demand, tailored to specific empirical data, to be analysed. All in all it can be said, that the modular approach of IMPEx lends itself perfectly to future extensions towards cloud computing and related technologies.
Project Results:
This chapter details the main results in science and technology for every RTD related work package of IMPEx as well as WP5 that was leading the implementation of the IMPEx Portal.
Data and Models Environment (DaME)
The major goal of Work Package 2 consisted in providing a user-friendly environment where the models and their results can be compared with the data of space missions and used jointly for scientific research and technological applications. The key objectives were:
• To design and implement an integrated environment of tools enabling the visualization of data originating from simulations, observations and models.
• To define and use standards for data and time table exchange between tools (CDPP-AMDA, CDPP-3DView, CLWeb and simulation databases).
• To define and use several methods of data discovery, aimed at helping the user fine-tuning the observations/models/simulations comparison.
• To provide a variety of data representations in multiple dimensions.
• To initiate collaborations with similar infrastructures in order to avoid duplication of efforts.
Integrated environment of tools
The first task of WP2 was the definition of an architecture facilitating the communication between scientific software tools, computational services and databases, as depicted in the figure below.
Figure 3: Overview of the IMPEx architecture and flow of information.
The implementation was facilitated by the adoption of widely used standards like SOAP, REST and HTTP, or IVOA SAMP and VOTable. In order to exchange data sets, a detailed description of their contents is necessary. This is achieved with the definition of metadata. Since a metadata model for observations already existed, it has been extended to take into account data coming from simulations. This task was conducted in collaboration with WP3.
Interfaces and protocols for data and time tables exchange
It is now possible to browse the contents of PMM and HMM catalogues in CDPP-3DView, CDPP-AMDA and CLWeb, using a file containing all the metadata necessary to access HMM or PMM data. A common format has been defined for this file, which is provided by all simulation databases (SMDBs). Every IMPEx tool must parse this file to display an interface for searching and getting data.
It is also possible to exchange data between CDPP-AMDA and CDPP-3DView using the SAMP protocol, widely used by astronomers. When CDPP-AMDA and CDPP-3DView are active on the user’s desktop, CDPP-AMDA can send data in VOTable format via a SAMP-Hub. These data can be scalars, vectors or spectrograms. CDPP-3DView implements a SAMP listener, which is waiting on a signal from CDPP-AMDA. When data are available in the SAMP-Hub, CDPP-3DView creates the related graphical objects and displays them in the scene. If these data are not in the same coordinate system, a transformation of coordinate system is performed.
Figure 4: Browsing IMPEx data in CDPP-3DView (with filters).
New methods for data discovery
New methods for data discovery were investigated and implemented; the selection of a run for comparison with observations is composed of two steps:
1) The first step is via the HMM and/or PMM simulation and model databases (SMDBs) catalogues browsing interface, implemented in the accessing interface. The files containing the metadata are provided by the SMDBs; currently FMI, LATMOS and SINP. They are parsed and displayed entirely as a hierarchy of data.
2) The second step consists in “filtering” the hierarchy by applying one or several criteria to the simulation input parameters contained in the metadata files, to reduce the size of the hierarchy of data, according to these criteria, which are selected by the user. An interface enabling this feature is provided in CDPP-3DView. It is e.g. possible in CDPP-3DView to shorten the list of runs to those related to a specific body (e.g. Mars or Earth).
The other way, an analytic (minimization) method of best fitting run among those available in HMM and GUMICS archives, has been investigated and implemented. This ordering method is called “N-index”. An SMDB that archives simulation runs, calculates this N-index and provides it through a web-service. This method, described below, is called getMostRelevantRun.
The best use of this method in the IMPEx infrastructure is to select data from GUMICS runs in CDPP-AMDA which already provides users with search and filtering capabilities in its workspace, but these capabilities are not applicable for a huge number of items, as it is the case for FMI/GUMICS, which provides about one hundred thousand runs.
Figure 5: Find the most relevant GUMICS Run in CDPP-AMDA.
Data representation in multiple dimensions
One of the great achievements of IMPEx is the possibility to display new types of scientific data, coming from observations as well as simulations in CDPP-3DView. Users can now display time series along the trajectory of spacecraft, field lines or flow lines, and cuts in two dimensions. Several use cases using these new capabilities in CDPP-3DView are presented below, including the comparison of simulated and observed time series, the synchronization of 2D and 3D displays, 2D-cuts, magnetic field lines, velocity flow lines, and spectrograms.
Figure 6: CDPP-3DView - Magnetic field vector and field lines around Saturn simulated by SINP, overlaid with CASSINI/MAG data, provided by AMDA (source PDS).
Figure 7: Impact of a CME (2012/06/16) visualized with in-situ data in CDPP-3DView. Cluster 1 and Geotail from CDPP-AMDA, simulations from FMI-GUMICS.
Figure 8: Display of several types of simulation data along Rosetta orbit in CDPP-3DView (hybrid model from FMI for comets).
Figure 9: Display in CDPP-3DView of several types of simulation data around Mars (MEX). Hybrid model from LATMOS.
Figure 10: Spectrogram of simulation data from FMI along Venus Express trajectory.
In CDPP-AMDA, it is now possible to display on the same 2D plot, time series of local or remote data coming from observations and remote data resulting of simulations, as depicted in the figure below:
Figure 11: CDPP-AMDA plot displaying comparison between observed data in the Martian environment (by the MEX spacecraft) and simulated ones (panels without the MEX tag). Magnetic field (6th panel from top) is absent from MEX instrumentation and can only be inferred from simulation models.
Collaboration with similar infrastructures
IMPEx WP2 has initiated several collaborations with similar infrastructures:
CCMC: The goal of the IMPEx-CCMC prototype was to render CCMC results more easily accessible to the wider community, by providing access, visualization and analysis via CDPP tools (CDPP-AMDA and CDPP-3DView). Another goal was to check whether the IMPEx Data Model, used to describe simulations to be compared with observations, was applicable to SMDBs that did not participate in IMPEx. The prototype was limited to one type of simulation, namely BATSRUS (MHD), and one run with data interpolated along the trajectory of several magnetospheric spacecraft. It focused mainly on access from CDPP-AMDA and CDPP-3DView. CCMC provides interpolation (in the 3D box) of physical quantities (fields and plasma parameters) along spacecraft trajectory as time series. They can now be directly compared to in-situ observations in CDPP-AMDA or CDPP-3DView.
EuroPlaNet-RI: An interface between IMPEx and EuroPlaNet-RI, another FP7 partner has been implemented, via a protocol defined by EuroPlaNet-RI, to access an IMPEx related database (CDPP-AMDA) from an external tool (TopCat). EuroPlaNet-RI has defined a common data model, EPNCore and a protocol, EPNTap. Both are used to give access to planetary science data. IMPEx has developed an interface between one of the participating tools, CDPP-AMDA and a search client designed and implemented by VO-Paris, in the framework of EuroPlaNet-RI. The same interface may be used between CDPP-AMDA and TopCat, a tool developed in the framework of IVOA (International Virtual Observatory Alliance) to analyze scientific data. These interfaces are based on EPNCore and EPNTap. In addition to that, IMPEx has created a translation mechanism between the IMPEx data model, which is based on SPASE and EPNCore. Every resource described within the IMPEx metadata trees can then be consumed by the EPN search client or by TopCat. Scientists familiar with TopCat are now able to use its numerous options to efficiently analyze and display a lot of datasets from space missions provided by CDPP-AMDA and simulations provided by IMPEx.
Extension of the CDPP-AMDA database of observations
A lot of datasets from space missions, relevant for observations/simulations comparisons, were added to the CDPP-AMDA database during IMPEx. They include data from missions to Mercury (Messenger), Venus (Venus Express, PVO), Mars (MAVEN, MGS, Mars Express), the Moon (Artemis) or the Giant planets (CASSINI, GALILEO, VOYAGER 1 & 2, PIONEER 10 & 11, ULYSSES), Comet Churyumov-Gerasimenko (Rosetta), and Comet Halley (Giotto).
Links to remote databases were also established for Venus (Venus Express) and the Moon (Artemis).
Outreach and educational products
A tutorial, demonstrating the access to data coming from observations and simulations with CDPP-AMDA and CDPP-3DView has been provided (see D2.15 as well as the demonstrator section of the IMPEx website, https://sites.google.com/site/impexfp7/). With this step-by-step guide users are able to visualize data related to the Earth magnetosphere, coming from several origins:
• Observations from CDPP-AMDA database
• SINP paraboloid model for the magnetic field
• FMI MHD model (GUMICS code)
• BATSRUS (MHD) run at CCMC
Hybrid and MHD Models (HMM)
WP3 (Hybrid and MHD Models (HMM)) was one of the two work packages which formed the IMPEx modelling sector. The basic objective of WP3 was to enable IMPEx to access modelling codes for simulation of plasma environments around planetary objects. Further it included the preparation of efficient interfaces for the operation of these codes and to work with their results. The modelling core of WP3 consisted of FMI’s hybrid (HYB) and magnetohydrodynamic (GUMICS) models and LATMOS’ hybrid model. Hybrid models were used for the study of solar system objects environments and the magnetohydrodynamic model for the terrestrial magnetosphere. The modelling infrastructure of WP3 builds on computational resources operated at FMI, as well as their related data bases and services.
The computational modelling activity in WP3 fed directly into WP2 in a way that the models and their data outputs were linked to the IMPEx infrastructure, which also included a large archive of space mission data (CDPP-AMDA) and a 3D visualization tool (CDPP-3DView).
The key products obtained in WP3 are the following:
• Defining the content and the structure of the IMPEx methods (*.wsdl files) and the structure of the simulation metadata (tree.xml files).
• IMPEx web service interfaces, which enable direct access to SMDB and, therefore, combining observation and simulation data.
• Web interface Hybrid Web Archive (HWA), which provides a user friendly interactive web access to FMI’s HYB and GUMICS model results and to its 3DHWA visualization facilities.
• Web interface LatHyS, which provides a user friendly interactive web access to LATMOS’ hybrid model runs results and to its visualization facilities and analysis tools.
• Open source IMPEx method scripts, which provide an example of how IMPEx methods have been implemented by FMI. These scripts help other model providers to build own
IMPEx compatible web services.
• Open source HYB model. This enables users to make own runs with the FMI’s HYB model and, importantly, check rationale and correctness of the used algorithms.
• 3D HWA visualization tool, 3DHWA, which helps FMI’s HMM SMDB users and users of the IMPEx methods to analyse runs in the HWA.
• Open source MATLAB scripts, provided by FMI, which provides MATLAB users an easy usage of the IMPEx methods
• Creating HMM modelling demonstrators for public education
The IMPEx Data Model, Interfaces, protocols and catalogues
A large effort on the definition and the implementation of the IMPEx Data Model has been conducted within WP3, together with WP2 and WP4. This effort has been consolidated in a new deliverable D3.14 not originally planned. Most of the development work for this data model concerned the description of the simulation runs and models. For this, we created new resources: SimulationModel, SimulationRun, SimulatedRegion, SimulationProduct, SimulationDomain, SimulationTime, and SimulationType (also see Figure 37).
We also made use of xml elements already defined in SPASE, to describe the simulation temporal and spatial parameters, as well as the targeted celestial body of the simulation. In all our developments we tried to limit the introduction of new terms or elements to facilitate the parsing of the metadata by automated tools (the larger the number of terms is, the more difficult it is for tools to handle them). An example of these new resources is presented in Figure 12.
Figure 12: Graphical representation of the SimulationModel resource
HMM environment data for CDPP-AMDA database
This effort consisted in linkage of HMM models and input/output data of modelling runs to the CDPP-AMDA database - it was performed in coordination with WP2. The developed data model link provides access to various modelling tools and data products.
The work has contributed to the definition of the tree.xml file which describes all HMM simulation runs and the accessible variables thereof. The file is publicly available from IMPEx’s data services.
Each simulation provider is responsible for providing the tree.xml on its server. A data model (standardized xml vocabulary and grammar) has been defined to standardize the tree.xml format. The tree.xml file describes run inputs and all simulation data, and provides links to the pre-computed data.
The IMPEx Simulation Data Model was designed as an extension of the existing SPASE Data Model, which is widely used for space physics databases. This compliance with standards ensures an integration of simulation databases in IMPEx, along with observation databases and an easier expendability of the IMPEx framework.
All methods have been defined for the SMDB. In HMM the methods are the following:
1. getDataPointValue: Returns a physical variable or variables at given point(s) from a model database.
2. getDataPointValueSpacecraft: Returns a physical variable or variables along the orbit of a spacecraft from an SMDB.
3. getDataPointSpectra: Returns the energy spectra at given point(s) from an SMDB.
4. getDataPointSpectraSpacecraft: Returns the energy spectra along the orbit of a spacecraft from an SMDB.
5. getSurface: Returns a physical variable or variables on a plane which normally is along the coordinate axis of an SMDB.
6. getFieldLine: Makes vector field tracing as e.g. magnetic field line tracing or flow line tracing.
7. getParticleTrajectory: Makes particle tracing (provided only by FMI).
8. getMostRelevantRun: Returns an ordered list of runs that have input values as close as possible to the provided set of values.
9. isAlive: returns the status of the database (Alive or Down).
One important task was to add ~1e5 GUMICS Earth magnetosphere result files into the tree.xml file in a way that keeps the size of the xml-file reasonably small and hence fast to use. The effort was successful and the method can be used efficiently on even such a large number of files. This also demonstrates that IMPEx methods and protocols are scalable and can handle in the future hundreds of thousands of missions and simulation result files. The HYB model was used to provide simulation results for Venus, Mars and comet 67P/Churymov-Gerashimenko. It further supports data analysis of ESA’s Venus Express, Mars Express and Rosetta missions, respectively.
HMM environment data for visualization
This task provided the outputs of HYB and MHD models for the CDPP-3DView visualization software provided by WP2. It is performed in coordination with WP2, and is connected to WP3.
The available data products and computational methods to access HMM simulation runs from CDPP-3DView were defined in the tree.xml and methods.xml files.
In addition to the CDPP-3DView visualization, a purely web based 3D visualization tool 3DHWA, was developed to provide a 3D tool to visualize HYB and GUMICS simulation runs in HWA. User picks up an interesting run from the HWA in a normal way. The user can then pick up plotting option “3D” (Figure 13) which enables interactive web based 3D analysis of the data.
Figure 13: An example of how 3D plots can be visualized via HWA (http://hwa.fmi.fi/) by its 3D visualization tool 3DHWA. The user can choose the “3D”page from the “Data Options”.
After that, a user can choose the analysed 3D region and how many planes are plotted.
Moreover, the user can add the orbit of the spacecraft at a given time range. The orbit is obtained by a web service provided by CDPP-AMDA (see Figure 14).
Figure 14: An example of a 3D plot obtained at HWA (http://hwa.fmi.fi/) by its 3D visualization tool 3DHWafter the user has pressed the button “Plot Data”, as shown in Fig. 13. The window makes it possible for a user to analyse data at various planes and to study how the data is related to the orbit of the spacecraft.
HMM-WWW interface for external users and infrastructures
The goal was to provide the HMM modelling tools, being further developed within IMPEx, for those users who operate data sets and services that are not part of the IMPEx core infrastructure (e.g. Europlanet-RI, HELIO, CCMC etc.), as well as for users who may want to apply HMM for basic research or for technological development.
Simulation runs stored in HWA can be accessed and searched by users. In Figure 15 an example is shown, where simulation runs are categorized according to the object and simulation run name - the physical and numerical parameters are also shown. Plot options and data download functionalities provide access to the simulation run results.
Simulation runs on HWA can also be searched by their input parameters - Figure 16 shows an example of that.
An example of access to simulation results (HWA’s 2D plotting facility) is demonstrated in Figure 17 (left). The variables to select for downloading are shown in Figure 17 (right). As noted before, in addition to 2D plots the HWA offers also a possibility for 3D plots.
The HMM-Direct Access Mechanism (HMM-DA) is realized using web service technology which is a standardized method of communication between two computers/applications over the internet. The interface to the service is described in a machine-processable format called WSDL (Web Services Description Language), which is an xml language. The interface defines all services (methods) that the server provides along with all necessary input and output format descriptions. Other systems interact with the web service by exchanging messages using the SOAP protocol.
HMM modelling demonstrators for public education
This effort included activities to provide a set of HMM modelling demonstrators, organized in the form of interactive web based educational services addressed to general public. The modelling demonstrators provide tutorials and explanations of the computational and physical backgrounds, making them ready for use in public educational institutions (schools, universities, planetariums, etc.).
Descriptions of the HYB modelling platform are available as a file and via the IMPEx Website (see https://sites.google.com/site/impexfp7/home/hmm). Furthermore, as illustrated in Figure 18, the HWA main page (http://hwa.fmi.fi) contains several links to the webpage which has been prepared for HMM modelling demonstrators for public education purposes. The pages describe the HYB (Figure 18 b) and GUMICS (Figure 18 c) models, the physical basics of the models, example of results, publications etc. Moreover, several videos were made and distributed via YouTube which describe and demonstrate the properties of the HWA in English as well as Finnish language, including code and usage of Matlab scripts. Last but not least, the HYB code, Matlab and FMI’s IMPEx web service scripts are made openly available at GitHub (see https://github.com/fmihpc/impex-tools).
Figure 18: An illustration of the access of video demonstrations and descriptions of the models. The HWA main page a contains links to the description of the HYB model (b), GUMICS model (c) and YouTube demonstration videos about the HWA and open source MATLAB scripts (d).
Complementary catalog at LATMOS
The goal of this effort (also see task 3.7) was to complete the catalogue of hybrid simulation runs by creating a new SMDB and linking it to CDPP-AMDA and CDPP-3DView tools.
In addition to the development of the infrastructure of simulation databases and services, LATMOS has made a significant contribution to the definition of the IMPEx Data Model (see D3.14).
Figure 19: Screenshot of the LATMOS SMDB web interface.
The simulation catalogue is publically accessible and some documentation concerning the simulation model and the web services can be found on the LATMOS website (http://impex.latmos.ipsl.fr) - a screenshot showing a partial description of the simulation runs available is presented in Figure 19.
The catalogue includes simulation runs for various celestial bodies such as Mars, Mercury and Ganymede. This database is characterized by the provision (via open access) of 3D data cubes and the full set of data products, in a way that enables the whole community to access the various simulated quantities. The full plasma package released in the catalogue consists in electric and magnetic fields, electron number density and temperature, plasma bulk speed and all ion species information (density, velocity and temperature). A detailed description of the data-products is provided in D3.5. Information concerning the URL of these pre-calculated files is indicated in the tree.xml file. Tools like CDPP-AMDA or CDPP-3DView parsing the tree.xml have the information about the pre-calculated files (see D3.7).
Interoperability of the LatHyS website with VO-tools
In addition to the archive of full 3D cubes simulations results, we archive and provide pre-computed 2D plane cuts for all data products. These pre-computed products can also be downloaded; it gives a preview of the simulation results. Moreover, LATMOS web-interface proposes connection to VO tools, like Topcat (Tool for Operation Catalogues And Tables, http://www.star.bris.ac.uk/~mbt/topcat/ ), using the SAMP protocol (Simple Application Messaging Protocol, developed for the IVOA community http://www.ivoa.net/documents/REC/App/SAMP-20090421.html). We can use SAMP as a hub and route all messages between clients. It allows exporting and displaying 1D and 2D data products on these tools.
A simple and interactive connection between LATMOS SMDB and Topcat has been developed on LatHyS (the LATMOS web-interface).
LATMOS’ Webservices
2D or 1D data products which are not stored in the LATMOS SMDB, e.g. a 2D cut that is not included in the set of pre-computed archived 2D cuts, can be requested by IMPEx tools (e.g. CDPP-AMDA, CDPP-3DView, CLWeb or the IMPEx Portal) through web services, as defined in the IMPEx Protocol.
The eight released LATMOS web services are:
1. getFileURL: this web service returns URL/granules for 1D data product.
2. getDataPointValue: a generic method that can be used to return parameters for 0D (a given point), 1D (along a curve/trajectory), 2D (in a plane) or 3D (inside a volume).
3. getDataPointValueSpacecraft: this Method interpolates the physical simulation parameters along a given spacecraft.
4. getSurface: this method is called via CDPP-3DView. It is used to generate a meshgrid and compute interpolation for one or several parameters.
5. getFiledLine: this method is called via CDPP-3DView. It computes field or flow lines for requested footprints or passing through the S/C track.
6. getDataPointSpectra: This method returns ion spectra for a requested position or a list of positions.
7. getDataPointSpectraSpacecraft: this method returns ion spectra along the spacecraft track (the spacecraft is requested from the user).
8. isAlive: this method returns the status of the database (available or not).
Figure 20 shows a schematical description of one of the web services. A full documentation of LATMOS web services is provided online as xml documentation (see http://impex.latmos.ipsl.fr/Methods_LATMOS.html) and through the IMPEx Technical Documentation section that can alse be accessed via the IMPEx website (see http://impex-fp7.oeaw.ac.at/fileadmin/user_upload/pdf/ListofWebservices_for_LATMOS_v1.0.pdf).
Figure 20: Schema of input parameters required for one web service.
Outreach and Educational resources
The description of the model, the hybrid formalism and the archive is presented in the IMPEx Demonstrators webpage (see https://sites.google.com/site/impexfp7/) additional documentation can be found on the LatHyS website (http://impex.latmos.ipsl.fr/doc/Hybrid_model_documentation.pdf) and in D3.12.
Two tutorial videos on the interoperability of LatHyS and VO-Tools (Topcat, CDPP-AMDA and CDPP-3DView) are posted on the IMPEx Website (http://impex-fp7.oeaw.ac.at/84.html) and have been created in synergy with the Europlanet-RI project.
Paraboloid Magnetospheric Models (PMM)
The WG4 (SINP) main scientific task was to upgrade the generalized paraboloid model of the planetary magnetosphere, so that it could be used for the on the fly calculations in the IMPEx framework. The model is intended for description of the magnetospheric dynamics, taking into account the intrinsic planetary magnetic field, the magnetopause current magnetic field, the tail current system magnetic field, the magnetodisc magnetic field, the ring current magnetic field, and the interplanetary magnetic field, penetrated from the solar wind. Such modular structure allows us to represent the magnetic field inside the magnetospheres of Mercury, Earth, Jupiter, and Saturn.
Based on the Earth's magnetospheric space missions’ data, and the missions to Mercury (Messenger), Jupiter (Ullysses, Galileo), and Saturn (Cassini) systems, the important features and dynamics of these planetary magnetospheres were discovered. The Hubble Space Telescope (HST) images of the Jovian and Kronian aurora also brought a new finding related to the outer magnetospheres structure.
For the Earth's magnetosphere, the time dependent functions of the global magnetospheric parameters were found on the basis of the solar wind monitors’ data and geomagnetic indices. As a result, the current state of Earth's environment is represented as a function of solar wind parameters and geomagnetic activity level.
The Cassini measurements were compared with the modeled magnetic field in Saturn’s magnetosphere. The Kronian auroral UV images obtained by the Hubble Space Telescope, together with the Cassini solar wind magnetic field data gave us the dependence of Saturn’s outer magnetospheric and ionospheric dynamics on the solar wind dynamic pressure and the interplanetary magnetic field.
During the Messenger flybys Mercury’s dipole moment was estimated. A northward dipole offset about a half thousand km was found by minimization of differences between the model calculations and Messenger’s magnetometer data.
The scientific studies of the SINP group allowed the development of the model, which can be implemented in the IMPEx framework as the basis for real-time calculations of the magnetic field in the different magnetospheres.
The key objectives and milestones in the course of the project for the IMPEx group and in particular for the team at SINP were:
● Development and elaboration of the Paraboloid Magnetospheric Models for Earth, Mercury, Saturn and Jupiter.
● Scientific publications on the results of the PM Models development by SINP.
● Establishing a general approach to a unified access to the data producers (FMI, LATMOS, SINP, CDPP) by members of the IMPEx team.
● Support for SOAP web service technology as communication technique between the data producers and the IMPEx Portal, visualization tools etc.
● Support for NetCDF and VOTable.
● Defining main functionalities of data producers: value in points, along a line, on a surface, in a cube and magnetic field lines.
● Implementation of the web services defined for the planets: Earth, Mercury, and Saturn.
● Elaboration and improvement of the web services, together with web service consumers (IMPEx Portal, CDPP-3DView, CDPP-AMDA).
● Development of the SINP visualization tool prototype based on WebGL technology.
● Constant coordination with the IMPEx team during the project’s lifetime.
● Preparation of documentation for models and services.
Web services
Two approaches to the model usage have been developed via SINP web services: get – i.e. access to database and interpolation of pre-calculated data and calculate – i.e. direct calculations in real-time. Following the list of services, implementing these two principles:
getDataPointValue, getSurface, calculateDataPointValueFixedTime, calculateDataPointValue, calculateDataPointValueSpacecraft, calculateFieldLine, calculateCube, calculateDataPointValueMercury, calculateCubeMercury, calculateFieldLineMercury, calculateDataPointValueSaturn, calculateCubeSaturn, calculateFieldLineSaturn, calculateDataPointValueJupiter, calculateCubeJupiter, calculateFieldLineJupiter
Paraboloid Magnetospheric Model of the Earth
The magnetic field can be calculated in the solar magnetospheric coordinate system only inside Earth's magnetosphere, in the region -40 < x < 20 Re; -30 < y,z < 30 Re, which is bounded by magnetopause, represented by a paraboloid of revolution. Inside the magnetopause of the paraboloid model (PM) of the Earth's magnetosphere (Alexeev et al., 2003), the magnetospheric magnetic field of each large scale current system is determined by an analytical solution of the Laplace equation for the magnetic field scalar potential. The magnetic field component, normal to the magnetopause is assumed to be zero. The model represents the magnetic field inside the magnetosphere as a sum of the internal planetary magnetic field, given by the IGRF2015 model and the external one (Bm), represented by a superposition of the magnetic fields of the ring current, Br, the tail current system including the currents across a tail, and their closure currents on the magnetopause, Bt, the Region 1 field-aligned currents, Bfac the magnetopause currents screening the dipole field, Bsd, and the magnetopause currents screening the ring current magnetic field, Bsr:
Here BIMF is the interplanetary magnetic field partially penetrated into the magnetosphere. The model input values are the key parameters of the magnetospheric current systems, which represent their location and intensity:
● the geomagnetic dipole tilt angle Ψ,
● the magnetopause stand-off distance R1,
● the distance to the inner edge of the tail current sheet R2,
● the magnetic flux through the tail lobes Φ∞,
● the ring current magnetic field at the Earth's center br,.
The time dependent model parameters are calculated by the empirical data (solar wind density (n), velocity (v), Dst and AL indices, interplanetary magnetic field B-components (IMF_B) and by the current date/time using special submodels (Alexeev et al., 2003) optimizing parameter dependences on the specific sets of empirical data. Input model parameters could be specified by the user or taken fully or partially from the OMNI database for a given time moment (StartTime input parameter).
getDataPointValue
“get-” methods are based on the approach of interpolation values in the precalculated 3D-cubes in order to return values in the requested points. 3D-cubes are a set of three-dimensional arrays, which contain magnetic field vectors, calculated in points of a grid around planets. The tree.xml file contains a catalog of these pre-calculated 3D-cubes.
The web service getDataPointValue can be used to get a magnetic field vector, calculated by PMM in chosen points and inside a volume.
Magnetopause dimensions were determined by input model parameters of the corresponding run.
Figure 21: Components of the magnetic field vector calculated in points with getDataPointValue (Topcat).
calculateDataPointValueFixedTime
Calculations realized through “calculate-” services allow the model runs in real-time. The web service calculateDataPointValueFixedTime can be used to get a magnetic field calculated by PMM in chosen points in one time moment (i.e. a ‘static’ picture).
Figure 22: Magnetic field vector magnitude calculated in points with calculateDataPointValueFixedTime (Topcat).
calculateDataPointValue
The web service calculateDataPointValue can be used to get a magnetic field (calculated by PMM) in chosen points and in different time moments for each point (i.e. a ‘dynamic’ picture). Model parameters are retrieved from the database for each time moment (each point) - can be used for magnetic field calculation along a spacecraft trajectory that is provided by the user.
Figure 23: Magnetic field vector magnitude calculated in points with calculateDataPointValueFixedTime (Topcat).
calculateDataPointValueSpacecraft
The web service calculateDataPointValueSpacecraft can be used to get a magnetic field along the spacecraft trajectory in the chosen time period calculated by PMM. The spacecraft trajectory should be inside Earth's magnetosphere. Input model parameters are obtained from the database for each position of spacecraft and time moment within the chosen time period. Spacecraft trajectory is obtained through the CDPP-AMDA web service. Calculations can be made along the spacecraft's trajectories, provided by CDPP-AMDA.
Figure 24: l.t.r: Magnetic field calculated along Themis-A trajectory (CDPP-3DView). Example of the magnetic field visualization (Topcat). Magnetic field calculated along Themis-E and Cluster4 trajectories (CDPP-3DView).
calculateFieldLine
The web service calculateFieldLine can be used to get field lines (calculated by PMM), starting in chosen points and in different time moments for each point, including the magnetic field along these lines. Magnetic field lines starting points should be placed inside the valid region. Input model parameters are retrieved from the database for each field line. The user determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 25: Magnetic field lines, calculated from the given starting points with calculateFieldLine (Topcat).
calculateCube
The web service calculateCube returns a magnetic field, calculated by the Paraboloid Model in grid points of a cube. Time moment, cube boundaries and grid sampling can be set by user.
Figure 26: Visualization of the magnetic vector field magnitude, calculated with calculateCube, in the cube -20
getSurface
The web service getSurface can be used to generate a meshgrid and perform a magnetic field interpolation on the grid between values from a pre-calculated PMM 3D-cube. Input: a point and a normal vector to the requested plane as input parameters.
Paraboloid Magnetospheric Model of Mercury
The magnetic field can be calculated only inside the Hermean magnetosphere that is implied to be bounded by magnetopause, represented by paraboloid of revolution in the region -5.5 < x < 2.6 Rm; -4.1 < y,z < 4.1 Rm in the Hermean solar magnetospheric coordinates system. The paraboloid model (Alexeev at al., 2010) represents the magnetic field as a sum of the Hermean internal magnetic field RINT and magnetic fields of the tail current, Rt, the magnetopause currents, Rsd, and the interplanetary magnetic field partially penetrated into the magnetosphere of Mercury, RIMF:
Model input parameters (all are optional):
● BD - dipole field strength on the equator of Mercury,
● Flux - magnetic flux at the polar cap open field line region,
● RSS - subsolar magnetopause distance in Mercury radii (2439km),
● R2 - the distance to the inner edge of the tail current sheet,
● DZ - northern displacement of the dipole relative to the center of Mercury,
● IMF_B - components of the Interplanetary Magnetic Field penetrated into Mercury’s magnetosphere (in the HSM coordinate system).
calculateDataPointValueMercury
The web service calculateDataPointValueMercury can be used to get a magnetic field (calculated by PMM) in chosen points.
Figure 27: Components of the magnetic field vector calculated with calculateDataPointValueMercury (Topcat).
calculateCubeMercury
The web service calculateCubeMercury returns a magnetic field, calculated by Paraboloid Model, in grid points of a cube with chosen boundaries and sampling.
Figure 28: Components of the magnetic field vector calculated in the cube -7.5
CalculateFieldLineMercury
The web service calculateFieldLineMercury calculates magnetic field lines, which start in chosen points. The user determines the length of the field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 29: Magnetic field lines, calculated with calculateFieldLineMercury, along the spacecraft Messenger trajectory on the 2009/09/29 (CDPP-3DView).
Paraboloid Magnetospheric Model of Saturn
The magnetic field can be calculated only inside the Kronian magnetosphere, which is bounded by magnetopause, represented by paraboloid of revolution (in the region -1200 < x < 40 Rs ; -500 < y, z < 500 Rs in the Kronian Solar-Magnetospheric coordinate system). The paraboloid model (Alexeev at al., 2006) represents the magnetic field as a sum of the Kronian internal magnetic field BINT (Burton et al.,2010) and magnetic field of the external sources Bm, which is a superposition of the magnetic fields of the magnetodisc, Br, the tail current system including currents across the tail and their closure currents on the magnetopause, Bt, the magnetopause currents, Bsd, and the interplanetary magnetic field partially penetrated into the Kronian magnetosphere, BIMF:
Model input parameters (all are optional):
● BDC - magnetic field at the magnetodisc (MD) outer edge
● bt - minus Z-component of the magnetic field at the tail current sheet inner edge
● RD2 - distance to the inner edge of the magnetodisc,
● RD1 - distance to the outer edge of the magnetodisc,
● R2 - distance to the inner edge of the tail current sheet,
● Rss - magnetopause stand-off distance,
● IMF_B - components of the Interplanetary Magnetic Field penetrated into the magnetosphere in the KSM coordinate system.
For each input model parameter a user can set a value manually or take a value by default.
calculateDataPointValueSaturn
The web service calculateDataPointValueSaturn returns a magnetic field (calculated by PMM) in chosen points with different timestamps but one set of parameters for all the points.
Figure 30: Magnetic vector field magnitude calculated with calculateDataPointValueSaturn (Topcat).
calculateCubeSaturn
The web service calculateCubeSaturn returns a magnetic field, calculated by the Paraboloid Model, in grid points of a cube. Cube boundaries and grid sampling can be set by user.
Figure 31: Components of magnetic vector field in the cube -20
calculateFieldLineSaturn
The web service calculateFieldLineSaturn calculates magnetic field lines that start in points chosen by the user who also determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 32: Magnetic field lines by calculateFieldLineSaturn, along Cassini trajectory (CDPP-3DView).
Paraboloid Magnetospheric Model of Jupiter
The magnetic field can be calculated only inside the Jovian magnetosphere, which is bounded by magnetopause, represented by paraboloid of revolution (in the region -450 < x < 150 Rj; -300 < y, z < 300 Rj in the Jovian Solar-Magnetospheric coordinates system). The paraboloid model (Alexeev and Belenkaya, 2005) represents magnetic field as a sum of the Jovian dipole magnetic field BINT (System III model) and the magnetic field of the external sources Bm, which is a superposition of the magnetic fields of the magnetodisc Br, the tail current system including tail currents and the closure currents on the magnetopause Bt, the magnetopause currents Bsd, and the interplanetary magnetic field partially penetrated into the Jovian magnetosphere BIMF:
Model input parameters:
● BDC - magnetic field at the magnetodisc (MD) outer edge,
● bt - minus Z-component of the magnetic field at the tail current sheet inner edge,
● RD2 - distance to the inner edge of the magnetodisc,
● RD1 - distance to the outer edge of the magnetodisc,
● R2 - distance to the inner edge of the tail current sheet,
● Rss - magnetopause stand-off distance,
● IMF_B - components of the Interplanetary Magnetic Field partially penetrated into the Jovian magnetosphere in the JSM coordinate system.
For each input model parameter a user can set a value manually or take a value by default.
calculateDataPointValueJupiter
The web service calculateDataPointValueJupiter returns a magnetic field (calculated by PMM) in chosen points with different timestamps, but one set of parameters for all the points.
Figure 33: Magnetic field vectors calculated in points of the Galileo spacecraft trajectory on the 2002/11/01 with calculateDataPointValueJupiter (CDPP-3DView).
calculateCubeJupiter
The web service calculateCubeJupiter returns a magnetic field, calculated by the Paraboloid Model, in grid points of a cube. Cube boundaries and grid sampling can be set by the user.
Figure 34: Components and slices of magnetic vector field in the cube -100
calculateFieldLineJupiter
The web service calculateFieldLineJupiter calculates magnetic field lines that start in points chosen by the user. The user determines the length of field lines and step size along them. The sign of the step size parameter defines which way field lines go: positive value - along the planetary magnetic field vector, negative value - against the magnetic field vector.
Figure 35: Magnetic field lines, calculated with calculateFieldLineJupiter, along Galileo trajectory on the 2002/11/01 (CDPP-3DView).
The IMPEx Portal
The IMPEx Portal has been fully developed at the IWF in Graz in the course of the bachelor and master thesis of Florian Topf (also see chapter 1.4.2). The IMPEx Portal has been defined as part of WP5 in RP2, in accordance with the Project Officer Antonio Fernandez-Ranada Shaw, since one of its main purposes is to attract new users to IMPEx and to introduce the capabilities and hence the potential of IMPEx in a user friendly and comprehensive way.
The development phase of the IMPEx Portal was concluded successfully in RP4. All major features derived from various requirements obtained e.g. by the advisory boards have been implemented and validated. The portal is now a central element of the IMPEx environment and a one-stop-solution for any interested parties and potential users. It allows obtaining a quick, practical insight into available tools as well as methods on a lower level of the system. The portal map (see Figure 36) connects all IMPEx enabled tools and also allows browsing, querying and transferring data from one of the attached SMDBs to e.g. CDPP-AMDA via SAMP. All queries are conducted in a unified internal registry that is updated on a daily basis, pulling in meta data from all attached SMDBs. It also conducts an “is alive” test at start-up and hence is an effective monitoring tool of the IMPEx environment including all attached components of the system.
Access to the IMPEx Portal is completely free and no user credentials must be provided for access. Nevertheless users can store selections made and results obtained via methods calls in the local browser storage for later access. This way data can be saved for later use, without the need of authentication. The portal is developed using HTML5/CSS3 technologies and can be operated on desktop computers as well as mobile devices without restrictions. To support the latter it can adapt to the actual screen size, offering an optimal experience for on all devices
Figure 36: The IMPEx Portal showing tab of the "portal map".
As can be seen in Figure 36, the portal offers several different sections via a tapped interface, these are briefly described in the following:
• Section “Map”: The portal map is the main interface shown per default, detailing all IMPEx enabled tools and SMDBs and allowing to query data, call methods, store results, transfer data etc.
• Section “Config”: Allows accessing the IMPEx configuration file as xml as well as json. The configuration is also parsed and displayed in a human readable form.
• Section “API”: This section offers a REST based interface to the IMPEx Protocol. Further it offers methods to retrieve the configuration file and query the internal (meta-)data registry.
• Section “Support”: This section offers a simple way of contacting the IMPEx team with regard to specific (technical) questions. At least one expert of each team is part of the list of recipients - the form offers select boxes to specify to which tool or part of IMPEx the question is related to (consequently also indicated in the mail’s subject line). All questions posed will be listed, thus building up a (searchable) FAQ in the future.
• Section “User Manual”: A comprehensive manual as a PDF for all types of users of the IMPEx Portal. The manual also features examples and step by step descriptions, including numerous screen shots of real use cases. The document is available at:
https://sites.google.com/site/impexfp7/home/impex-portal-user-manual
• Section “Tool Docs”: This section provides the QuickGuide to IMPEx Tools, which was motivated by feedback obtained from the advisory boards in RP3. The QuickGuide gives a brief overview of IMPEx and its available tools, and thus directs the user to the tool that best fits his/her needs or use case. The document is available at:
http://impex-fp7.oeaw.ac.at/fileadmin/user_upload/pdf/IMPExToolsandtheirUsability.pdf
Potential Impact:
This chapter details the potential impact of the IMPEx project on planetary science in general and R&D activities in this field, including the socio-economic and the wider societal implications of the project so far as well as the main dissemination activities and exploitation of results.
Impact of technological developments of IMPEx
One of the main achievements of IMPEx-FP7, the definition of a comprehensive data model applicable on numerical simulations in the area of space plasma physics and the plasma environments of planetary objects, is expected to have a significant impact on planetary science. Until the advent of the SPASE simulation extensions (which have been directly derived from the IMPEx Data Model, also see Figure 37 and D3.14) numerical models mostly used proprietary data structures and metadata descriptions to store and process data, obtained via modelling runs. This of course considerably increased the complexity for modelling and research groups to exchange data and solve scientific and technological tasks in a concerted effort, involving data from multiple sources. Since data needs to be consistent and described in a homogenous way, each team more or less had to translate back and forth between the data structures used by the different group respectively – this is of course not necessary any more, as soon as all teams agree to use one common data model. However, this again is only feasible, once a format exists that satisfies the requirements of all parties involved and allows to describe all relevant aspects of the data in relation to the respective scientific investigation at hand.
Figure 37: A diagram of the IMPEx Data Model - the first complete data model of its kind that has been finalized, documented and fully released for public use, allowing to fully describe modelled space plasma data.
The IMPEx team is convinced that the SPASE simulation extensions fulfil the basic requirements of a wide range of research carried out in space plasma physics. Moreover, since it is a living definition, the model can be extended in the future to include further elements that will enable new users to describe aspects required for their specific research. To this behalf, processes have been defined within the SPASE group, and this way new additions to the data model are regularly discussed in web conferences and added to the definition, once agreed by all parties and stakeholders involved. Hence, it is expected that this new data model based on SPASE (already widely used in the field of observational data in the field of space plasma physics) will foster the exchange of data between different modelling groups and even more so open up data archives formerly only used and even accessible within specialized mission teams to the wider scientific community.
Since communication among experts and scientists is of course crucial in research, the impact of a common data standard, once adopted by a sufficiently big number of researchers, archives and institutions cannot be underestimated. The less time is lost in administrating and distributing the data, and the easier it is to compare results obtained by different groups, the more efficient research can be carried out. This is of course achieved by enabling automated tools and applications to handle data in a consistent and transparent way, thus freeing users from having to deal with the specifics of different formats. The more tools are enabled to access and process data, the more users are reached and the bigger the impact hence is. To further facilitate the interoperability of tools, IMPEx created a common protocol that covers a broad range of requirements. In IMPEx this was achieved primarily by defining a set of web services (i.e. the IMPEx Protocol) that allows different tools as e.g. CDPP-AMDA, CDPP-3DView and various SMDBs to exchange data and call functionalities taking these data, stored compliant with the IMPEx Data Model (i.e. SPASE simulation extensions which is an inherent component of the protocol), as input. This way all users of this set of IMPEx enabled tools can access a common data pool and researchers, students, post docs and the public in general have access to all information and functionalities of the IMPEx environment.
As with the data model, the set of web services can be extended as well, to cover requirements in the future that were not thought of or even viable during the project’s inception or lifetime. Moreover, the definition can also be taken as a basis for proprietary extensions by other institutions, while preserving backward compatibility, taking advantage of the fact that many problems one is faced with when designing a coherent set of web services to handle plasma data, have already been successfully solved in IMPEx. It should also be noted that the IMPEx Protocol is also able to handle models for real-time calculations, as well as pre-calculated cube data stored in an SMDB (see Chapter 1.3.3 for further details on real-time methods).
Since SOAP is used for the basic implementation, tools and web apps all over the world can harvest the available functionalities more or less automatically and bring them into their respective software landscape, hence making them accessible and workable for their specific community. This way the idea of IMPEx will also spread across the web and it can be expected that it will encourage other projects to follow a similar path or to seek cooperation with institutions that are part of the original project team - see D5.3 for more information on projects and institutions that are already collaborating with IMPEx. If users run into access restrictions at certain tools or SMDBs connected to IMPEx, they can simply contact the respective institution providing the service and ask for a free account. In any case the philosophy behind IMPEx is to offer as much data and functionalities for free as possible, which is an approach that is highly beneficial for scientific endeavours in general. The more tools and applications in the scientific world are able to interoperate without restriction, the easier it is to distribute information and collectively tackle problems that are difficult to solve for a single entity, researcher or institution. Following this approach has been shown to be very successful in the context of IMPEx, and it is to be expected that this advantage scales up very well to the scientific community as a whole. After all, as already briefly discussed in Chapter 1.2 there are also other initiatives, as e.g. IVOA that are following this approach with great success.
Information technology naturally is an essential part of modern science, and building solutions that are designed to provide certain minimum standards with regard to usability and stability is a time consuming task that requires professional software developers to be engaged for considerable time spans. It can’t be stressed enough that a lot of resources are freed up and thus made available for pursuing the actual research, when (IT related) solutions already implemented and validated at one institution can be re-used at other institutes and projects, either solving the task altogether or providing a stable basis on which further, more specialized solutions can be built upon. In a nutshell, IMPEx thrives to provide the common language that allows this close interaction to take place on a technical level as well.
With regard to access restrictions, it should also be stressed that all functionalities available at the IMPEx Portal, i.e. all methods offered by participating nodes (tools and SMDBs) are fully accessible without any credentials required at all, and hence free of charge. The same is true for most of the data that can be browsed and downloaded via the portal interface, or the respective SMDB providing the data in the first place. However, there are of course (observational) data sets, e.g. offered by the tool CDPP-AMDA that are not yet fully released by the respective PI, and thus require users to log-on to the system with specific credentials, in order to be able to access them. However, this is completely outside of the influence of the project members of course.
Socio-economic and wider societal implications
Software development is a costly endeavour since applications and software environments satisfying up-to-date standards with regard to usability and stability require considerable effort in terms of design, development as well as testing and validation. Thus, purely scientific institutions rarely have the time and resources available to develop customized software in a coordinated way, which leads to a scattered software landscape in the field of science tools that often cannot satisfy the true needs of its user communities. Here projects like IMPEx can make a valuable contribution, since dedicated funds are available and professional software development can be conducted via e.g. sub-contracting of development companies as e.g. GFI in IMPEx engaged by CNRS for the development of the visualization software CDPP-3DView. Apart from the implementation of ready to use applications, also the definition end extension of standards is a vital part of modern software development, keeping in mind that without standardization and the definition of appropriate interfaces no true interoperability of different applications can be achieved. As with writing code, the definition of sound, stable and comprehensive standards and interfaces requires a considerable coordinated effort, involving all or as many stakeholders as possible, in order to produce a meaningful definition that will actually be leveraged in future development efforts. IMPEx has not only produced ready to use software, but also put a lot of thinking, time and effort into the framework itself, i.e. the IMPEx Data Model and the IMPEx Protocol (including further definitions as e.g. the IMPEx Configuration etc.).
This broadens the base of stable implementations and definitions available in order to build specialized, heavy tailored solutions on top. The latter can then be again performed much more easily by institutions that have a strict scientific focus. Here of course the flexible architecture of IMPEx, relying on a standard data model and a set of web services that have been designed with the principle of generality in mind, provides a valuable contribution to the (planetary) science community. More efficient software tools naturally lead to better scientific results and more productiveness by minimizing administrative and purely technical tasks, allowing scientists to focus on the actual investigation and scientific questions at hand.
Another important aspect are the numerous tutorials (video, speech and textual), and the technical documentation produced during the project. The videos can be used as a quick and comfortable way of getting an overview of the possibilities offered by IMPEx, while the documents (as e.g. the Hybrid Model Demonstrators or the IMPEx Tutorial for CDPP-AMDA & CDPP-3DView) provide all required information and resources to then deepen the knowledge in specific (scientific and technical) areas, or with regard to concrete tool support needed. The IMPEx Portal which is designed to be a one-stop-solution for newcomers to IMPEx is also covered by a dedicated and comprehensive tutorial that allows potential users to quickly get started. These materials can also be used by students and senior classes as well as teachers and at university level in general, in order to prepare classes and gather material for talks and presentations. It is the hope that the educational materials and tools provided will help to encourage pupils and students to pursue a career in space physics. The visual tools as e.g. CDPP-3DView are particularly suitable in order to attract attention and generate interest through presentations etc. All tutorials can be accessed on the IMPEx website at the IMPEx Demonstrators section at https://sites.google.com/site/impexfp7/ - IMPEx tutorial videos can be viewed directly at the IMPEx Videos section, as well as from the HYB Code channel available on YouTube – the latter featuring videos for FMI’s hybrid and MHD codes, explaining their theoretical background as well as their usage via IMPEx and the SMDB interface provided by FMI.
An important socio economic impact of IMPEx is the fact that many positions at the participating scientific institutions were created, and in particular post-docs and students were able to work for the project and gain valuable (first) experiences and insights into new technologies and IT related approaches.
Figure 38: The IMPEx Team at PMC 36 in Moscow (May 2014). Florian Topf (2nd from right, bottom row), Manuel Scherf (3rd from right, middle row) and Tarek Al-Ubaidi (first from right, top row).
At IWF also a project management position was supported during project execution, held by Tarek Al-Ubaidi, who brought many years of experience in the field of professional software development, web development and knowledge management into the project. Also at IWF, Florian Topf was engaged for development work – in the course of his employment Florian Topf (supported by Tarek Al-Ubaidi) worked out a comprehensive concept for the IMPEx Portal, and successfully finished his bachelors and then also his master thesis focusing on the IMPEx portal, scientific tools in general, and the benefits of functional programming in this field. Tarek Al-Ubaidi also served as a supervisor for the master thesis of Florian Topf. Finally Florian Topf also implemented the portal according to the design agreed by the team at a face-to-face meeting at the end of RP3 (in the course of PMC #36 in Moscow), using scala, an innovative run time environment and development platform, based on the functional programming paradigm. Also at IWF Manuel Scherf was engaged as a research assistant, supporting the team in the creation of tutorials and the editing of scientific material from a didactic point of view. Further Manuel Scherf was pivotal in validating software products that were created or extended in the course of the project, as e.g. the visualization software CDPP-3DView or the IMPEx Portal in general.
At SINP (Moscow State University) Lucy Mukhametdinova was engaged, starting at the beginning of RP3. She was responsible for supporting Vladimir Kalegaev in implementing, testing, bug fixing and last but not least documenting the extensive web service interface provided for the paraboloid model and the SMDB at SINP.
At FMI David Perez-Suarez did extensive experimentation with regard to workflow integration of the IMPEx toolset, in particular in connection with the scientific workflow system Taverna (also see periodic reports for RP2 and RP3) that was used to build prototype workflow support for IMPEx on the basis of the IMPEx Protocol.
AT CNRS/LATMOS Sebastién Hess was engaged as a post-doc in RP2 and RP3, in order to support the definition process of the IMPEx Data Model. Sebastién Hess subsequently took the lead in this process and made a decisive contribution to this effort that resulted in the successful finalization of the prototype version of the data model in RP3.
As already discussed in the course of technical impacts, IMPEx, through its very nature of being a software environment, providing the means for tools and services to communicate and exchange data, very well supports and even fosters the cooperation of different institutes, not only on a purely technical, but a scientific and educational level as well. Hence many new cooperative links have been either newly established in the course of the project, or deepened due to a now available (technical) infrastructure that allows to do things that were either impossible or of a too high effort prior to IMPEx. At the end of RP4 IMPEx has active links to the following projects and infrastructures (also see D5.3 for further details):
● SPASE (http://www.spase-group.org/)
● Europlanet/IDIS (http://www.europlanet-idis.fi/)
● HELIO (http://www.helio-vo.eu/)
● CCMC (http://ccmc.gsfc.nasa.gov/)
● ETFLA/Astronet (http://www.astronet-eu.org/)
● ASTRONET (http://www.astronet-eu.org/)
● VESPA (http://vespa.obspm.fr/)
The level and nature of cooperation is of course dependant on available funding, as well as the focus of the respective institution involved. While e.g. cooperation with ASTRONET is primarily of a scientific nature, the ties to the SPASE community are more or less purely technical. Links to CCMC and Europlanet were primarily driven by WP2, see chapter 1.3.1 for further details.
Further cooperative ties are built, as this report is being written. One of them concerns a technical as well as scientific link to UCLA; here the integration of simulation results is being prototyped and the respective data is already part of the IMPEx Configuration. Another link is being established to LESIA (http://www.lesia.obspm.fr).
The impact of these scientific and technological ties is often considerable, given that developments at one institution often drive or spin-off further developments at other institutions that would not have been possible or feasible otherwise without the groundwork being laid. By taking advantage of synergies, it is often possible to minimize the effort involved in following a certain research topic, or develop a solution for a highly specialized problem, specific to a given scientific use case.
Dissemination activities and exploitation of results
IMPEx has been presented at numerous scientific conventions and congresses, including the annual EPSC (European Planetary Science Congress, organized by the Europlanet community), the EGU (European Geosciences Union) and other important events, covering the field of planetary science. These events included poster presentations as well as oral presentations, hands on sessions and talks on various subjects of the IMPEx projects, ranging from technological to purely scientific topics.
In June 2012 Tarek Al-Ubaidi e.g. joined the Planetary Data Workshop for Users and Developers that took place in Flagstaff Arizona. The oral presentation that was received with great interest also resulted in a closer exchange with Dan Crichton (PDS Engineering Node Manager at NASA/JPL), with regard to efforts related to the (then on-going) definition process of the xml based PDS4 data model, to integrate modelled plasma data into PDS4, as the IMPEx simulation extensions are doing in the context of SPASE. Dan Crichton consequently also joined the IMPEx User Support Board in 2012 – please refer to Table 2 for a complete list of dissemination activities for the entire project duration (until June 2015). It is expected that IMPEx and its technological as well as scientific results will continue to be referenced extensively in publications and talks, in particular those held by members of the IMPEx core team (IWF, FMI, CNRS, and SINP). As an example this year’s EPSC taking place in October in Nantes, France, will include a poster presentation on possible follow up actions and projects, talking the ideas of IMPEx to the next level, also integrating cloud resources and advanced data discovery technologies.
Starting in RP2, also annual briefing sessions were held to inform the board members (USB and SSB boards, see D1.3 for details) of all the latest developments, and to gather valuable feedback that was consequently processed and henceforth strongly influenced further developments as well as the planning process. The live events held as face-to-face meetings at EGU in Vienna in RP2 (2013) and RP4 (2015), were public in the sense that any participant of EGU could take part.
Several press releases detailing the project goals and current status were written and subsequently published throughout the project. As captured in D5.2 of WP5, two articles were written by the IMPEx team (under the lead of IWF-OeAW) and published in the Parliament Magazine in RP1 and RP2 respectively. In the latter half of the project, the Vienna based PR company PR&D, specialized on the publication of scientific content, wrote an article in close cooperation with IWF-OeAW and with input and feedback from the entire IMPEx team. The publishing effort by PR&D turned out to be very effective; the first article in RP3 (titled Ready For Take-Off - Launch of New Data Model Boosts Space Science) reached over 60 confirmed coverages. The second article by PR&D published in RP4 (European Team Creates Universal "Language" for Space Science) reached 74 confirmed coverages (May 2015). The media outlets also included local newspapers (e.g. Kleine Zeitung) technical journals (e.g. Computerwelt or Deutsches Ingenieurblatt) and scientific publications (e.g. science 2.0). The articles by PR&D were available (and published) in German as well as English language. In RP4 also Finnish and Russian versions were prepared and published locally. The team at IWF-OeAW also prepared a publicly accessible directory that included several images that could be published along with the text provided. Since RP3 there is also an IMPEx Press Kit available from the website that can be used by journalists and other interested parties to gain further background knowledge on the project, its content and goals.
The IMPEx Website (also see chapter 1.5) also includes a video section that currently features six video tutorials on the following subject:
• IMPEx Tutorial Video - Tutorial demonstrating the combined use of the IMPEx Portal and several IMPEx enabled tools (IWF-OeAW, ~10 min.).
• Tutorial on interoperability of CDPP-AMDA, LatHyS and Topcat - this video was created by Europlanet in cooperation with IMPEx using funding from FP7/REA, and presented at EPSC 2013 (~7:30 min.).
• Tutorial on CDPP-AMDA, CDPP-3DView and Simulation Databases (SMDBs) - this video was created by Europlanet in cooperation with IMPEx using funding from FP7/REA, and presented at EPSC 2013 (~10:30 min.).
• Auroral Processes of Saturn - This video was created by Europlanet with funding from FP7/REA and presented at EPSC 2013 (~9:00 min.).
• Tutorial on IMPEx Matlab Tools (~9:40 min.)
• Tutorial on Hybrid Web Archive (~06:40 min.).
Further there are nine videos (mostly in Finnish language) available from the following HYB video channel, provided and maintained by FMI: https://www.youtube.com/channel/UC-auXDoJSbYBVrSsI2-NpPg The channel deals on various subjects surrounding the hybrid code (HYB) as provided by FMI, see Table 2 for further details.
The IMPEx Podcast has been part of the project from the start and issued 16 episodes in total over the whole project duration of four years. Each episode featured a guest from the wider (planetary) science community covering a topic that is related to the project. Many of the episodes featured members of the IMPEx core team, including the coordinator, deputy coordinator, scientific representative, work package and task leaders as well as members of the advisory boards. The podcast is an ideal way of becoming acquainted with the project, its goals and main achievements. All episodes are hosted and produced by Tarek Al-Ubaidi and can be listened to via the IMPEx website: http://impex-fp7.oeaw.ac.at/podcast.html. Also see Table 2 for a complete list of episodes including titles and featured guests.
The IMPEx Portal that has been implemented in the course of WP5 (dedicated to outreach and dissemination) is also an important instrument for attracting new users to the system. The portal can be accessed with any desktop or mobile device, capable of displaying and executing HTML5/JavaScript based web apps, and collects all relevant information at one spot. Users can enter the portal without having to provide any credentials and are free to browse data provided by SMDBs or to call methods of the IMPEx Protocol, supported by a graphical and easy to use interface. It is an attractive way of making oneself familiar with all relevant IMPEx capabilities (including the IMPEx Configuration, the REST based web service interface etc.) and hence become involved on a very practical level. This way the sometimes rather abstract definitions and concepts of IMPEx (in particular for non-IT savvy users) become much clearer, easy to grasp and fully leverage. Please refer to Chapter 1.3.4 for further (technical) details on the IMPEx Portal.
List of Websites:
The IMPEx website (also see Figure 39) is the central access point for all IMPEx relevant information. It provides access to comprehensive technical documentation, tutorials, videos and the IMPEx Podcast. Further there is general information about the team, links to publications & talks, articles and news on latest developments of the project.
The IMPEx website can be reached via: http://impex-fp7.oeaw.ac.at/
The website is administrated and moderated by Tarek Al-Ubaidi (tarek.al-ubaidi@oeaw.ac.at, +43 316 4120 673) and Manuel Scherf (manuel.scherf@oeaw.ac.at, +43 (316) 4120-672). For technical problems please contact Manfred Stachel (Manfred.stachel@oeaw.ac.at, +43 (316) 4120-412).
