Final Report Summary - SOLARNET (High-Resolution Solar Physics Network)
Executive Summary:
SOLARNET is a project aiming to bring together and integrate the major European research infrastructures in the field of high-resolution solar physics, in order to promote their coordinated use and development. The project involves all pertinent European research institutions, infrastructures, and data repositories.
Networking activities, access to first-class infrastructures and joint research and development activities have been covered under SOLARNET to improve, in quantity and quality, the service provided by this European community.
SOLARNET involves 32 partners from 16 countries: 24 EU research institutions, 6 EU private companies and 2 USA research institutions. The co-ordinating institution is the Instituto de Astrofísica de Canarias.
SOLARNET provides access to leading world-class ground-based research infrastructures, aims at equipping them with advanced instrumentation, and prepares for the next generation of high-resolution telescopes, with the European Solar Telescope (EST) as the most prominent facility.
The following actions have been carried out under the auspices of SOLARNET:
- Transnational Access to external European users.
- Enhance and spread data acquisition and processing expertise to the community.
- Increase the impact of high-resolution data by offering science-ready data.
- Provide unified access to pertinent data repositories with space and ground-based data.
- Foster synergies between various research communities by organising joint meetings.
- Train a new generation of solar researchers through schools and a mobility programme.
- Develop prototypes for new-generation post-focus instruments.
- Study local and non-local atmospheric turbulence and ways to negate its effects.
- Improve the performance of existing telescopes.
- Improve the design of future large European ground-and space-based solar telescopes.
- Lay foundations for combined use of facilities around the world and in space.
- Foster partnership with industry to promote technology transfer with existing networks.
- Disseminate activities towards society.
The project started on April 1st 2013, and the 3rd reporting period covers since April 1st 2016 to March 31st 2017. The progress achieved in this period is summarised below.
Project Context and Objectives:
WP10: Project Coordination and Management
This work-package has the following main objectives:
-Effective coordination and management of the whole project.
-Implementation of an effective and transparent management of the project, ensuring appropriate control and monitoring: schedule, budget & scope (milestones and deliverables).
-Effective communication among activities and teams.
-Monitoring and reporting of the overall progress and use of the infrastructures for access, assuring the timely delivery of high quality deliverables and milestones.
-Contractual and financial follow-up of the project.
-Coordination of dissemination activities.
In order to guarantee an appropriate management structure and administration of the Project and its tasks and activities, an elaborated management structure was set up
WP20: Integrated operation and exploitation of solar physics facilities and coordination with other research infrastructures
The focus of this work package has been to optimize the use of the existing solar facilities and research structures. To achieve this goal, we have worked on a number of objectives: (20.1) coordination and implementation of the Transnational ACCESS and Service program, (20.2) to make available proper data pipelines for existing instruments, (20.3) development of guidelines for metadata, aiming at a standard for archived solar data, (20.4) coordinated observations of our facilities with non-European and/or non-optical infrastructures, (20.5) study new ways of operating our telescopes, in particular, gather experience with performing observations in service mode.
WP30: Solar Physics Networking
The aim of this work-package has been to foster collaborations among different solar physics groups, promote the interaction and cooperation among researchers of different level of expertise, as well as to encourage and promote synergies with other fields. Some of the actions foreseen under this work-package have been: exploitation of ground- and space-based data; enhancement of collaborations with other communities and Projects; promotion of collaborations between the new generation of scientists and experienced researchers through short stays and training actions to acquire competences in relevant fields of solar physics.
sWP30.1: Meetings on solar physics
One objective of this sWP has been to put in contact different solar physics communities and researchers involved in different fields of research
sWP30.2: Mobility
SOLARNET supported the mobility of young researchers as part of the networking activities in this work-package. The mobility programme has been designed, as an additional aspect of the training program, to reinforce the contacts between different groups and to allow young researchers to begin early to establish international collaborations.
sWP30.3: Training (Schools and Thematic Workshops)
The aim of this task was the organization of summer/winter schools for PhD students and novel post-doc researchers on topics related to the development of new instrumentation for solar observations, diagnostic tools, hot solar research topics and fields of mutual interest for solar and stellar physicists. This task had the goal to organize 5 Schools and Thematic Workshops.
WP40: Innovation Towards Industry
The overall objectives of WP40 can be defined as follows:
-Identification and analysis of technologies, techniques and new concepts already in use at the forefront of solar physics’ developments with potential interest for other sectors.
-Dissemination of the innovative aspects and findings of the project towards industry, through the existing and highly successful industrial associations and related high-tech companies at EU level.
-Assessment of space technologies of most interest for solar physics developments, and identification of common needs and opportunities.
-Extension of the effort of collaboration between high-tech companies, universities and public organisations on solar physics and related fields, oriented to the transfer of knowledge using well proven methodologies.
-Increase of the potential for innovation of research infrastructures on solar physics.
Promotion of synergies with other I3 initiatives on this same topic.
WP50: Tools for innovative data handling: pipelines, databases and SVO (led by SU)
Two were the main goals of this work-package:
(i)Develop data-reduction pipelines for the most important European ground-based high resolution solar instruments. Enhancement of observational procedures for increased productivity and easier co-observing and combination of data. The pipelines will produce data and meta-data fulfilling the requirements of a Solar Virtual Observatory (SVO).
(ii)Set up a prototype for a SVO archive.
sWP50.1 Data Reduction Pipelines
The goal of sWP50.1 was to produce data pipelines for the major European solar instruments. In this context, a data pipeline is the software that calibrates and administrates raw data so that it can be scientifically analysed. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories
sWP50.1.2 Data Compression
The goal of this task was to evaluate 3D compression algorithms for solar spectropolarimetric data.
sWP50.1.3 Image restoration
There are two kinds of image restoration methods in regular use for high-resolution solar imaging, particularly data sets with multiple wavelengths and/or polarization states. They are called Multi-object multi-frame blind deconvolution (MOMFBD) and Speckle interferometry with Speckle deconvolution, e.g. with the Kiepenheuer-institute speckle interferometry package (KISIP). One task of sWP50.1.3 was to improve the accessibility of the codes for both methods. The second important task in this sWP was to compare the two methods.
sWP50.2 Solar Virtual Observatory
The primary premise for this work was to produce the SOLARNET “Document on standards for data archiving and VO”.
WP60: Advanced Instrumentation Development
This work-package aims at the development of new instrumentation for existing solar telescopes, enhancing their capabilities and scientific potential. Some of the novel concepts and designs explored in this WP may be applicable to future large-aperture telescopes, as well.
sWP60.1: Large Diameter Etalon development
The purpose of task sWP60.1 is to build a prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI), with minimal cavity defects. The work done in this sWP represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
sWP60.2 Image Slicer for 2D Spectroscopy
Integral Field Spectroscopy (IFS) is a technique that provides the spectra of all the points in a bidimensional field of view at the same time and under the same conditions using an Integral Field Unit (IFU). Its application offers information along three dimensions: X, Y, λ. There exist different alternatives of Integral Field Units (optical fibers, microlenses and image slicers). In this sWP the latter option is explored with a prototype construction and testing. Task sWP60.3 has developed the microlenses alternative (see below). The objective of sWP60.2 was to develop a prototype of an IFU based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope, to make solar observations obtaining the spectra of a 2-D region simultaneously and to validate the concept for the European Solar Telescope instrumentation.
sWP60.4 Fast Imaging Polarimeter
The goal of the Fast Solar Polarimeter (FSP) project was the development of a novel solar imaging polarimeter with an emphasis on significantly increased polarimetric accuracy and high spatial resolution.
WP70: Wavefront control: turbulence characterization and correction
This work-package aims at measuring the turbulence surrounding telescopes and at determining the measures that can minimise its effects on image quality.
WP70 is divided into 3 sWPs, with a total of 5 tasks, two related with adaptive optics issues (sWP70.1) one related to seeing measurements (sWP70.2) and the other two related to seeing effects minimization (sWP70.3):
sWP70.1. Adaptive Optics (AO)
Task sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
Task sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
sWP70.2. Atmospheric Seeing Characterization
sWP70.3. Local Seeing
Task sWP70.3.1 Application of CFD techniques for local seeing optimization
Task sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
WP80. Synoptic observations: Solar Physics Research Integrated Network Group (SPRING)
Anetwork of telescopes with a small aperture but a large field-of-view can provide useful data to prepare observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures. The goal of this work-package has been to define the scientific requirements of such a network and to analyse the technical alternatives that can deliver the required data products of interest.
WP90. Access Programme to key ground-based infrastructures
The goal of this WP is to provide access to external users to the facilities included in the project (THEMIS, SST, VTT, GREGOR, IBIS@DST and ROSA@DST).
WP100. Access to Science Data Centres. Space missions (led by UiO)
Access to the most demanded European Science Data Centre, providing data gathered by the solar satellite HINODE, and the Solar Dynamics Observatory (SDO), will be offered. As previously mentioned, this access to space-based data for solar research, offered for the first time under an I3 supported by the EC, will expand significantly the content of the Programmes for the high-resolution solar physics community supported in the past.
The Hinode Science Data Centre Europe, the Belgian Web Incessant Screening and the German Science Center for the Solar Dynamics Observatory, are the three facilities offering access and services under this WP101, WP102 and WP103.
The main efforts of this WP, concerning these web-based databases, will be to widely advertise these resources, to increase the number of EU researchers accessing to them.
Project Results:
WP10: Project Coordination and Management
Under this WP, the following activities have been carried out:
-Board and Executive Committees Meetings. All documents of these meetings are available at the private pages of the SOLARNET webpage.
-Organisation of meetings and events: 33 meetings or related events were organised.
-Distribution of the payments to the partners: All EU funds were readily distributed to the partners after reception at the IAC
-Follow-up of deliverables and milestones: A total of 25 milestones were achieved and 72 deliverables produced
-Regular interaction with SOLARNET partners and WP and sub-WP leaders.
-Support to the Transnational Access and Service Programme. 59 astronomers supported with travel and subsistence grants to carry out observations in situ in the Canary Islands telescopes.
-Support to activities related to the Solar Physics Networking: four meetings, five SOLARNET Schools and Workshops, SOLARNET Mobility of Young Researchers Programme.
-Outreach aimed at reaching the goal of promoting joint actions specifically designed to enhance the visibility of the project at international level and getting the targeted audience at large.
-Follow-up of the use of resources. The total human effort has reached approximately 104.58 % of the total amount of manpower dedication expected for the four-year SOLARNET period (689 p-m).
WP20: Integrated operation and exploitation of solar physics facilities and coordination with other research infrastructures
The main body of the work package is the Forum for Access and Services (FAS). At the meetings of the FAS all objectives of this work-package have been monitored and discussed. All telescope and instrument owners have formed part of the FAS, and participated at its meetings. The meetings have been open to all people involved in relevant SOLARNET work- packages.
The EAST TAC has allocated jointly all SOLARNET ACCESS time to the various telescopes and instruments. In order to adopt the EAST TAC to its new challenges, the FAS decided at its first meeting to assign as EAST TAC members one representative of each institutions that operate the telescopes/instruments handled by SOLARNET (GREGOR, VTT, SST, THEMIS, IBIS@DST and ROSA@DST), plus one representative of IAC and NSO. EAST approved this assignment at its General Assembly 2013.
The EAST TAC granted observing time in PI mode as well as in the ‘service’ mode. The PI mode is the traditional observing mode, in which the principal investigator and parts of his/her team goes to the telescope and performs the observations. The technical support is typically limited to the telescope and the instruments need to be operated by the observing team. The service mode is a novel observing mode in solar physics. In this mode the proposers describe the scientific objective and give a detailed explanation on what data they need. The data is acquired by a service team at the telescope following the requirements that the proposers have specified.
The EAST TAC announced the call for proposals for the observing seasons since 2013 to 2016. The calls were issued in ‘SOLAR NEWS’, which is subscribed by most Solar Physicists worldwide. Also, the EAST TAC informed all ‘known’ people that could be interested in the ACCESS program. In addition the Project office implemented a web page within the SOLARNET internet portal to promote the ACCESS program, and distributed a poster on the TAS. The proposal deadlines were in mid-January of each observing season. The number of submitted proposal was higher than expected. The EAST TAC checked the proposals for eligibility and contacted the telescope operators for checking the technical feasibility. The EAST TAC selected two independent referees who rated the proposals according to their scientific merit. Based on the scientific merit, the EAST TAC granted the observing time.
The ACCESS programme was most successful in being oversubscribed and bringing new users to the telescopes. In the course of the 4-year program, the oversubscription rate increased steadily. In 2016 - the fourth year - the ACCESS program was oversubscribed by 200% (26 proposals for 13 allocated projects). If only PI mode campaigns are considered, the oversubscription rate is even higher. It amounts to 250% (22 proposals for 9 allocated projects). During the whole project, the EAST TAC allocated a total of 68 observing campaigns with 466 observing days. 353 users from 18 countries were awarded with observing time and 59 received a travel & subsistence grant to perform the observations. 30 of 59 of them were new users. The figures above show some statistics of the access programme.
Presentations and discussions on the four consecutive FAS meetings brought together all the data pipeline experts from the SOLARNET telescopes, as well as US data scientists that are involved in the data pipelines of the upcoming 4m solar telescope DKIST in Hawaii. Having brought together this community is a major success of WP20. The exchange of expertise in the design of data pipelines between the different groups manifests a most valuable result of the WP20 endeavour. Some aspects of this undertaking are summarized in the 'Final report on data pipeline guidelines'. The concept for a data pipeline needs to be distinguished, depending on the type of instrument: each type of instrument needs its own characteristic design.
A document to define a standard for the metadata in Solar Physics has been produced. The standard has evolved continuously and the document has been updated whenever considered relevant. The latest version can be downloaded at http://sdc.uio.no/open/solarnet-20.3/. In the community it is known as the SOLARNET standard for metadata and is most relevant to the community world-wide. The team of the US 4m solar telescope DKIST who is designing the concept for their data pipelines is also taking advantage of 'our' European SOLARNET standard.
In this sub-work package we have achieved another major breakthrough: the ingestion of ground-based high-resolution spectro-polarimetric data into the SVO prototype. This was made possible, because during the SOLARNET project, data pipelines and data archives adopted the SOLARNET standard and data archives for various SOLARNET telescopes and instruments were developed. For example, the near-IR spectro-polarimeter GRIS@GREGOR now offers a quick look data archive that provides science-ready data for external users: http://archive.leibniz-kis.de/pub/gris/. The GRIS instrument data pipelines provides the necessary metadata that is ingested into the SVO prototype, such that the GRIS data is fully searchable by external scientists. A similar data archive is provided by BBI@GREGOR, GFPI@GREGOR, and the Italian IBIS@DST data. Their metadata is also ingested into the SVO prototype.
In dedicated campaigns, facilities world-wide were coordinated to perform a joint scientific observation campaign. Progress in solar science relies on multi-instrument and multi-wavelength observations assisted by spectro-polarimetric observations with large field-of-views and long temporal coverage. Within this task, a number of such campaigns was performed and evaluated, using ground-based high-resolution telescopes in optical, near-IR, and radio wavelengths. Theses campaigns were supplemented by high-resolution and synoptic space-based missions like SDO, IRIS, STEREO, and Hinode.
New images taken with the Atacama Large Millimeter/submillimeter Array (ALMA) in Chile have revealed otherwise invisible details of our Sun, including a new view of the dark, contorted centre of a sunspot that is nearly twice the diameter of the Earth. The images are the first ever made of the Sun with a facility where ESO is a partner, with a substantial contribution of Dr. Roman Brajsa (HVAR) who acknowledges funding from SOLARNET. The results are an important expansion of the range of observations that can be used to probe the physics of our nearest star. The ALMA antennas had been carefully designed so they could image the Sun without being damaged by the intense heat of the focussed light. The corresponding press release can be found at http://www.eso.org/public/news/eso1703/ .
Prior to SOLARNET solar observing campaigns were executed in PI mode. A Principal Investigator travels to the telescope and performs the observations with a very limited level of technical assistance. This led to a distinct scientific career of an 'observer', since those PI needed to be acquainted with the optics and the instruments of a telescope, and how to use the sub-systems. As these telescope systems became more and more complex the observing task can no longer be performed by a solar physicist whose expertise focuses on solar physics. Therefore, novel observing modes needed to be developed in which a team of instrument experts carry out the observations and calibrate the data while the solar physicist analyses the data to the benefit of science. In the novel queue observing mode, observing proposals are evaluated and ranked by a time allocation committee, and executed by dedicated telescope staff.
During SOLARNET we gathered experience in operating solar telescopes in queued service mode performing dedicated campaigns at DST, SST, and GREGOR. Observing campaigns were queued according to scientific merit and executed based on several parameters like, e.g. target availability, seeing conditions, and instrumental configuration.
Valuable insights in practical realization issues were gained which will direct the implementation of the queued service mode at the planned European Solar Telescope (EST).
WP30: Solar Physics Networking
This work-package is divided in three sub-Workpackages:
-sWP30.1 Meetings
-sWP30.2 Mobility
-sWP30.3 Training (Schools and Thematic Workshops)
sWP30.1: Meetings on solar physics
Four meetings were organised by SOLARNET.
The 1st SOLARNET Meeting “Synergies between ground- and space-based solar research” was held in Oslo on 5 – 8 August 2013 (http://folk.uio.no/matsc/oslo-13/index.html ) together with the 3rd EAST / ATST Meeting. The meeting had 67 participants. The on-line Proceedings of the Meeting are available at http://folk.uio.no/matsc/oslo-13/program.html. Exemption of the registration fee was given to 26 young participants.
The 2nd SOLARNET Meeting "Solar and stellar magnetic activity" was held in Palermo, Italy, on 2-5 Feb. 2015 (http://www.astropa.inaf.it/Solarnet2015/Solarnet2015.html). Specific aims of this meeting were to review the current understanding of magnetic fields in the Sun and similar stars and to discuss future directions of research. The meeting was attended by 61 participants. The proceedings of the meeting are available at http://www.astropa.unipa.it/Solarnet2015/Proceedings/Proceedings.html. Exemption of the registration fee was given to 12 young participants.
The 3rd Solarnet Meeting “Solarnet III / HELAS VII / SpaceInn: The Sun, the stars and solar-stellar relations” was held in Freiburg 31 August – 4 September 2015 (http://www.iac.es/congreso/solarnet-3meeting/) together with the 7th International Meeting of the European Helio- and Asteroseismology Network (HELAS) and its project SpaceInn (Exploitation of Space Data for Innovative Helio- and Asteroseismology). The goals of the conference were to review the state of knowledge in helioseismology, which studies the interior of the Sun through observations of the waves observed at the surface, and the application of its tools and techniques to other stars, in the so called asteroseismology. The meeting had 84 participants. In addition to the plenary sessions, four splinter meetings were held. The meeting was covered in Twitter. All posts related to the conference can be found via #cid_solarnet3_2015. In order to reach out the broad scientific community and the public, all material presented at the conference is made available on the Open Access internet platform www.science-media.org. Financial support, in form of waiver of the registration fee and/or travel support was given to 14 young participants.
The 4th Solarnet Meeting “The physics of the Sun from the interior to the outer atmosphere” was held in Arrecife, Lanzarote (Spain) from 16th to 20th January 2017, during the Third Reporting Period. The purpose of this conference was to provide a coherent picture of the Sun as a single physical system playing all the underlying physical processes measured and observed in the solar atmosphere to date. The meeting had 143 participants. Financial support, in form of waiver of the registration fee and/or travel support, was given to 26 young participants. The meeting was covered in Twitter. The posts related to the conference can be found via #solarnet4m. The presentations and the posters can be downloaded at the following link:
http://www.iac.es/congreso/solarnet-4meeting/pages/talk-and-poster-download.php
sWP30.2: Mobility
SOLARNET supported the mobility of young researchers as part of the networking activities in this work-package. The mobility programme has been designed, as an additional aspect of the training program, to reinforce the contacts between different groups and to allow young researchers to begin early to establish international collaborations.
This task was supervised by the Mobility Evaluation Committee (MEC), a group especially created within SOLARNET, which evaluated and selected the submitted applications. The program supported travel, accommodation and subsistence costs up to a given amount per week. The application was always open and a deadline was set every six months. At that moment, the MEC met and decided, based on a combination of curricula and quality of the project, the young students whose visit could be funded. In order to obtain a better distribution of participants in the framework of the gender aspect, the poster advertising the last announcement included a statement inviting explicitely young female scientists to apply. The Mobility program, characterized by 7 Calls during the period March 2013-March 2016, has been a definite success, with a high number of applicants (45). One of the big successes of the program has been to use the available funds to cover more young researchers than initially expected.
Some statistical figures are:
-20 visits in 7 calls (5 women)
-range of duration: 4 – 14 weeks
-total weeks: 165
-average duration: 8.3 weeks
-average cost per student: 2440 €
-average cost per week: 251 €
-total costs 49826,29 € (planned 35200 €)
-5 – 6 papers in preparation/submitted/accepted
It is also worth to stress that the applicants to the last Call were all young female scientists, ensuring that the gender issue in the end was more balanced than before the last Call. All funded young researchers and most of the host institutions that they have visited have submitted brief reports summarizing the work carried out during the stay. This was one of the deliverables planned for SOLARNET. The high level of the collaborations clearly demonstrates the success of the program.
sWP30.3: Training (Schools and Thematic Workshops)
The 1st SOLARNET School was held in Wroclaw (Poland) from March 24 to April 4, 2014 (http://school.astro.uni.wroc.pl) and was dedicated to PhD students and novel post-doc researchers who wanted to broaden their knowledge on solar physics. In addition to the lectures, several "hands-on" sessions were planned, coupled to the appropriate lectures, in order to allow the participants to develop their practical skills. There were 40 applications for the school participation submitted to the Scientific Organizing Committee and 20 young researchers were finally selected. During the School, some lectures on complementary skills were provided, in order to describe the practical aspects in the work of researcher. The presentations and the materials provided by the lecturers are available at the following webpage: http://school.astro.uni.wroc.pl/materials.php . During the School each student gave a short presentation on her/his scientific work (around 15 minutes); all the presentations are available at http://school.astro.uni.wroc.pl/presentations.php . In parallel with the school, the SOLARNET workshop "Radiative processes in the Sun and stars" was also organized in order to allow the young researchers to participate in splinter sessions and to discuss their research with experienced scientists. The main aim of the workshop was to present and discuss the recent scientific results concerning different aspects of solar and stellar phenomena.
The 2nd SOLARNET School "Ground- and space- based solar instruments" (see http://www.astro.sk/SOLARNET_2ND_SCHOOL/solarnet) was held in Tatranska Lomnica (Slovakia), in October 5-16, 2014. The number of applications was 19 and 12 students were selected to attend the School. The lectures, carried out by experienced scientists, were aimed at giving students knowledge on current and future ground- and space- based solar instruments, observational strategies, post-focus instrumentation and data acquisition. The hands-on sessions were carried out also at Lomnický štít Observatory on the Double Solar Coronagraph and Coronal Multi-channel Polarimeter. There was information presented on the European Solar Telescope (EST) project and on the FP7 SOLARNET project, as well as lectures on complementary skills. In parallel with the school, the Workshop: "Methods in high resolution and synoptic solar physics" and 2nd SPRING (Solar Physics Research Integrated Network Group) were also organized in the same place, Slovakia (see the page http://www.astro.sk/SOLARNET_2ND_WORKSHOP/ for a detailed description of the program).
The 3rd SOLARNET School on “Solar Magnetic Fields: Modeling and Measuring Techniques” was held in Granada (Spain) between the 18th and the 23rd of May, 2015. The goal of this school was to provide PhD students and early career potsdocs with a solid background in solar spectropolarimetry, radiative transfer of polarized light and diagnostic techniques. The programme combined both theory and hands-on activities to achieve that goal. Lecture slides, as well as the necessary inversion codes for the hands-on exercises, were provided for downloading at the School’s website (http://spg.iaa.es/School). A total of 26 young scientists, comprising PhD candidates and early career postdocs from more than 10 countries, were selected to participate in the school (out of 32 applicants). Students were invited to stay one more week and attend the 3rd SOLARNET Workshop, to be held the following week, with the financial support from SOLARNET. The 3rd SOLARNET Workshop was held in Granada (Spain) since 25th to 28th of May, 2015. (http://granada-en.congresoseci.com/polarization2015/index).
The 4th SOLARNET School on “Solar MHD and Reconnection” was held at MSSL (Surrey, UK), 13th-19th of April, 2016 (http://www.ucl.ac.uk/mssl/solar/SOLARNET-4 ). The aim of this school was to provide students and early-career researchers with a solid grounding in Solar MHD and Reconnection using a combination of lectures and hands-on analysis sessions. The School was attended by 41 participants (16 women); further, 39 % of the speakers were women. Lecture slides (https://www.ucl.ac.uk/mssl/solar/SOLARNET-4/school-programme) were provided for downloading from the School’s website and are still available at the same link. The associated thematic Workshop on “Solar eruptive events: observations and modeling” (http://www.ucl.ac.uk/mssl/solar/SOLARNET-4/workshop-programme) was held between 20th and 22nd April. During the workshop the students attending the School were invited to present their research.
The 5th SOLARNET School on “Waves and oscillations in the solar atmosphere” and the thematic Workshop on “Heating Mechanisms in the Solar Atmosphere” were held at Queen’s University Belfast from the 25th of August to the 2nd of September 2016 (https://star.pst.qub.ac.uk/wiki/doku.php/public/solarnet5/start ). The School was attended by 25 early career researchers from 10 different countries (UK, Germany, Austria, Belgium, Norway, Sweden, Spain, USA, Algeria, Georgia. Among these, 11 attendees were women. Lecture slides, as well as the exercises and the necessary inversion codes for the hands-on activities, were provided for downloading from the School’s website (https://star.pst.qub.ac.uk/wiki/doku.php/public/solarnet5/start ) and are still available at the same link. The 5th SOLARNET Workshop was held in Belfast (UK) between the 31st August and the 2nd of September, 2016. The Workshop was attended by 48 researchers from 13 different countries (UK, Germany, Austria, Belgium, Italy, Norway, Sweden, Spain, USA, Algeria, Georgia, India, Bulgaria). Among these, 23 attendees were early career researchers and 15 attendees were women.
WP40: Innovation Towards Industry
To reach the objectives of this work-package, the following tasks were addressed:
- Identification and analysis of technologies, techniques and new concepts, through the development of a database for technology offers, consortium knowledge base and the technology needs identified by the solar astrophysics community
- Dissemination of the innovative aspects and findings of the project towards industry at international/European events, SOLARNET workshops, European workshops, ...
-Organisation of workshops related to technology transfer: (i) 1st Solarnet Technology Transfer Workshop linked with SOFT 2014 Fusion Conference (San Sebastian, Spain). It showed several technologies/needs with potential solutions/ collaborations within the fusion industry; (ii) 2nd Solarnet Technology Transfer Workshop was held during this period. Linked with LSW London Space Week, B2B and Summit Conference (London, UK). It showed a significant number of technologies/needs with potential solutions/collaborations within the space industry; (iii) o3rd Solarnet Technology Transfer Workshop between SOLARNET and ESA-ESOC (July 2016). Presentation of SOLARNET project and relation to EST. Presentation of EST/SOLARNET Technology Offers/Needs to ESOC. ESOC presented requirements related to future Optical Ground Stations.
- Assessment of space technologies: (i) Tecnalia has access to the ESA European Technology Transfer Network database; (ii) continuous monitoring of space tech-nologies for potential application in solar astrophysics community; (iii) mMonitoring of opportunities for transfer out from ground based solar telescopes to space based or other sector applications; (iv) Monitoring of Invitation To Tender (ITT) opportunities through the ESA EMITS portal.
- Dissemination of the project and specifically the technology offer and needs listings: (i) Numerous Spanish space industry meetings; (ii) Participation at International/European Events; (iii) Three SOLARNET Industrial workshops; (iv) Continued interaction with European networks (including ESA); (v) Fact finding mission to EC (Brussels); (vi) Continued development of national industrial contacts.
- Identification of future projects: Related to 3rd Industrial Workshop, WP40 has identified future potential ESA GSTP projects related to the development of the optical ground station that should be further assessed by members of the SOLARNET community for future funding opportunities: (i) Laser ranger (approx. budget 7-8 M€); (ii) Low Cost Meter-Class Adaptive Optics Communications Breadboard (1M€); (iii) Deep Space Low Cost 4m monolithic Optical Antenna for Day/Night Operations (400k€); (iv) 4m mirror concept (approx. budget 20M€).
- Promotion of synergies with other I3 initiatives on this same topic. The main contact with I3 initiatives has been with ASTERICS (www.asterics2020.eu). The aim of the collaboration was the management and exchange of large volumes of data related to ESFRI projects.
WP50: Tools for innovative data handling: pipelines, databases and SVO
To reach its objectives, the WP50 was divided into 4 tasks grouped within two areas corresponding to the main objectives:
sWP50.1: Data Reduction Pipelines
Task WP50.1.1: General pipeline work
Task WP50.1.2: Data compression
Task WP50.1.3: Image restoration
sWP50.2: Solar Virtual Observatory
sWP50.1 Data Reduction Pipelines
Task sWP50.1.1 General pipeline work
The goal of sWP50.1 was to produce data pipelines for the major European solar instruments. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories.
As input for the pipeline work, SOLARNET sWP20.2 issued guidelines with general recommendations. The most important of these was that output data should be in the FITS format with metadata included as standardised headers. Metadata are data that describe the circumstances of the observations and how the data have been processed. It was the task of SOLARNET sWP20.3 to produce recommendations for the FITS headers and they were issued in the corresponding report and much of the work addressed here was devoted to adapting pipelines to these recommendations. This is no small task. Being fully compliant with the recommendations means introducing 99 new keywords and their values while making sure that no other keyword is used in a conflicting way. At the same time auxiliary data such as wavelength scales would have to be treated and packaged according to new specifications. The SOLARNET recommendations include a self-declaration of its level of compliancy. Thus SOLARNET=1.0 means fully compliant and SOLARNET=0.5 means partially compliant (some fundamental keywords exist and there are no conflicts between other keywords and the SOLARNET recommendations). Perhaps the most important legacy of this effort is that the community has learnt how important these issues are and how they should be tackled.
The following is a summary of work being done for all instruments under the umbrella of SOLARNET.
GRIS (IAC): This pipeline is almost completely automatic, only demanding a minimum of input from the user. The output data are now very close to being SOLARNET compliant. As GRIS produces spectrograms, there is no image reconstruction technique involved.
BLISS/HiFI (AIP): When SOLARNET started, BLISS was a planned instrument for the GREGOR telescope. It was later cancelled but replaced by HiFI which is a context imager for blue light that is less ambitious than the projected BLISS. The SOLARNET Executive Committee approved that the software development for HiFI replaces that for BLISS. HiFI is one of the instruments that use the data pipeline sTools. HiFI metadata is conforming to the SOLARNET=0.5 level. A technical obstacle for reaching detailed compliancy is that the GREGOR telescope pointing recording has uncertainties, precluding exact automatic recording of the position on the solar disc. Image reconstruction using Speckle.
GFPI (AIP): The pipeline software is sTools which is also used for HiFI. Once the HiFI metadata treatment has been verified to comply with recommendations the procedure will be transferred to GFPI data. The work with GFPI data is more complicated than for HiFI. Image reconstruction using MOMFBD.
CRISP, CHROMIS (SU): CRISPRED, the pipeline for CRISP data, was developed during the first half of the SOLARNET. It has played an important role in solar physics for allowing several researchers to use CRISP. CRISPRED has active users in twelve institutes (as judged from the subscriptions for the mailing list for CRISPRED/CHROMISRED users). CHROMIS was installed at the SST in August 2016 and the first observations were performed during the following two months. This meant that much of the effort during the last months of the SOLARNET project was spent on producing its pipeline: CHROMISRED. This new pipeline was built with similar and partly overlapping software as the existing CRISPRED for CRISP. It has more than 50000 lines of code, not counting the image-reconstruction part. CHROMISRED is close to becoming fully compliant with the SOLARNET metadata standards (SOLARNET=1.0). After it has been found stable, CRISPRED will be taken to that level too as the two pipelines are merged. Both pipelines involve MOMFBD image reconstruction.
TRIPPEL (SU): This instrument is arguably the least standardised in the whole suite since observing campaigns make different choices for detectors and slit-jaw imaging setups. Thus many changes are made between different data sets, so much that it is not enough to set a few input parameters. The goal has been that such changes should be confined to a minimum number subroutines.
LARS (KIS): The instrument was upgraded in 2016 from a prototype to regular-use instrument. Starting with the 2017 observing season, LARS can be used also by non-expert observers. Metadata treatment is extensive and very close to being SOLARNET compliant though the formal declaration has not been made. Technical limitations of the telescope preclude automatic and precise recording of pointing coordinates on the solar disc.
TESOS (KIS): The metadata treatment was reworked and reached SOLARNET=0.5. Image reconstruction using Speckle.
ROSA (QUB): The work in the ROSA pipeline include introducing metadata treatment to the SOLARNET=0.5 level and writing a manual. The pipeline relies on speckle image reconstruction using KISIP.
IBIS (INAF): The IBIS pipeline exists in an INAF and an NSO branch. The INAF branch has undergone a metadata-treatment push to reach SOLARNET=0.5. In parallel, an archive for IBIS data has been built: http://ibis.oa-roma.inaf.it/IBISA/ . Apart from making IBIS data available to the community this way, this effort has also driven and motivated the pipeline work. Image reconstruction using MOMFBD.
TUNIS and MTR (CNRS): Level 0 metadata are almost completely compliant with the recommendations but have yet to be propagated to the fully reduced data.
It can be concluded that, thanks to the work in task WP50.1.1 there are now modern pipelines for all major European solar instruments. The pipelines have been raised to a new standard where they are more reliable and easy to use. This has led to higher productivity and more and better science. The work with conforming the pipelines to the needs of virtual observatories will allow the future development of such databases.
Task sWP50.1.2 Data Compression
Comparative tests carried out during the EST Design Study indicated that application of state-of-the-art 2D compression methods to solar spectro-polarimetric data leads to a final data volume that range between 1/2 (lossless compression) to 1/5-1/10 (lossy compression) of the original data volume. However, high-resolution spectroscopic and spectro-polarimetric solar observations consist of time series with small changes from one frame to the next. The correlation present in the spectro-polarimetric data-sets over all dimensions, spatial, temporal and spectral, suggests that the use of 3D compression algorithms applied to either (x,y,λ) or (x,y,t) data cubes may allow an enhancement of compression efficiency with respect to 2D methods, with no information loss.
Solar imaging data have, in general, peculiar characteristics, such as high S/N, continuous data on the FoV, etc., which are not always typical of other hyperspectral data-sets. The data-sets used in the Report on lossless JP3D compression of solar data-cubes are a partial description of the present variety of high resolution photospheric solar data, which will probably be maintained in the next years. The two data-sets are retrievable from https://www.fisica.uniroma2.it/~solare/en/?p=257 to allow anyone to compare the compression performances of different algorithms on the same data-sets.
To extend the analysis on chromospheric solar data, in Report on lossless JP3D compression of solar chromospheric data-cubes, we compressed and de-compressed a spectroscopic scan of a chromospheric line consisting in 80 repetition of a 21-point imaging scan of the CaII 854.2 nm line of the same Active Region. These data-sets include calibrated broad-band, narrow-band and spectropolarimetric observations of the photosphere and chromosphere. They also represent different types of targets: a Quiet Sun region at disk centre and an Active Region at disk centre. Therefore, they allow us to evaluate the performance of the compressing algorithm in different, but ’typical’ situations of solar observations: low-contrast images with localized small-scale magnetic fields, high contrast images with concentrated and diffuse magnetic field, highly dynamic chromospheric datasets, etc.
The compression methods studied constitute a selection among the most frequently adopted image compression procedures in a variety of fields of application; they exploit in different ways the properties of the Discrete Wavelet Transforms often coupled with standard entropy coders or similar coding procedures applied to the different bit planes in order to allow a progressive handling of the original image.
2-D lossless compression algorithms were able to reach a compression factor between 1.5 and 3 for the various types of solar images used. Among these algorithms, the one with the best compression ratios was the SPIHT, the one with the shortest compression+decompression times was the JPEG2000. Among the lossy 2D compression algorithms, the JPEG2000 had the best performance overall. The 3D compression algorithms allow us to gain a 2x factor over the 2D methods in the compression ratio with apparently no data degradation. Of course, this comes with a cost, namely the 3D compression is more demanding in hardware and more time consuming. Nevertheless, the evolution of the computing capacity is fast enough to allow us to perform the compression of large data-cubes with relative ease, and, above all, the computing capabilities are evolving faster than the storage or the data-transfer capabilities. Since the differences in the performances of these 3-D methods were within the errors, we choose JP3D because it is open-source and well documented. We applied loss-less JP3D compression to calibrated solar data-cubes of different types. The data-sets employed are a partial description of the present variety of solar data, which will probably be maintained in the next years. The performance of the 3D compression varies with the data-type: G-band are the most compressible (~3 BPV), while Stokes I are the less compressible (~7 BPV). The gain against a 2D compression varies from -50% to -30% of the original data volume. The gain in ordering data in [x,y,λ,t] or [x,y,t,λ] is apparently negligible. The JP3D algorithm is efficient in handling the larger files, with little differences in compressing a single large data-cube or several smaller data-cube.
As conclusion, the JP3D compression algorithm applied to either the (x,y,λ) or (x,y,t) allows an approximate doubling of compression efficiency on solar spectro-polarimetric data by 3-D algorithms with respect to 2D methods on same data. The performance of the compression varies with the data-type: G-band images are the most compressible (~3 BPV), while Stokes I are the less compressible (~7 BPV). We note that this performance is probably not enough to keep data volumes at manageable size (the target compression ratio should be of the order of 25x, i.e. ~1 BPV) therefore, we suggest to explore also the JP3D algorithm performances in lossy compression, for those cases where the preservation of all the information is not critical. We note also that, even considering an enhancement of the data processing capabilities in the foreseeable future, the compression time will be a crucial factor. In fact, the acquisition rates are likely to increase up to 100 1024×1024 pixel images per second and real time compression of the data-sets may not be feasible. This has to be taken in consideration in the observatory daily schedule.
Task sWP50.1.3 Image restoration
There are two kinds of image restoration methods in regular use for high-resolution solar imaging, particularly data sets with multiple wavelengths and/or polarization states. They are called Multi-object multi-frame blind deconvolution (MOMFBD) and Speckle interferometry with Speckle deconvolution, e.g. with the Kiepenheuer-institute speckle interferometry package (KISIP).
One task of sWP50.1.3 was to improve the accessibility of the codes for both methods. SU has made a fork of the MOMFBD code, and rewritten it extensively to make it easier to maintain (using more standard libraries in place of original code), adapted to the data format of the recently installed SST/CHROMIS instrument, and to implement a number of features that should improve the restorations. The new version, redux, is under version control with git and available for anybody at git://dubshen.astro.su.se/hillberg/redux. The speckle interferometry code KISIP is likewise under git version control and available from https://bitbucket.org/fwoeger/kisip.git. Both momfbd/redux and KISIP are integrated parts of data processing pipelines developed under SOLARNET, where standard parameter settings are provided by the pipeline maintainers.
Another important task in this sWP was to compare the two methods. While the comparison is interesting in itself, it also spurs everyone involved to understand the codes better. The method-comparison experiment at the 1st CASSDA-SOLARNET Workshop taught us that in order to interpret the results of such a comparison, it is important to have better control over the assumptions imposed by the calibrations and other processing done in different ways in the different data pipelines. Further efforts were needed to isolate image restoration effects from other steps in the pipelines.
A more carefully designed experiment was started. We installed the KISIP code as a drop-in replacement for MOMFBD in the CRISPRED pipeline, the data processing pipeline for the CRISP instrument at SST. This ensured that all processing before and after the actual image restoration is done in the same way (and in the best way known for that instrument), thus making the comparison more relevant. Using data from the SST, which is a refractor, removes uncertainties in, e.g. how the central obscuration of reflectors like GREGOR and the elongated pupil of VTT affect the image restoration.
Suitable data for this experiment were collected but we soon realized that the theory for one of the calibrations required for SI with AO compensated data had only been developed for AO systems where the deformable mirror is controlled with Zernike polynomials as modes. The SST AO uses control modes that are designed to take both the statistics of atmospheric turbulence and the mirror electrode pattern into account. We have worked to fill in this gap in the theory and a new paper on the subject will be submitted to Journal of the Optical Society of America A shortly.
sWP50.2 Solar Virtual Observatory
Work was concentrated on the data archive prototype linked to the SVO and the implementation of the recommendations in the pipeline work. The primary premise for this work is the SOLARNET “Document on standards for data archiving and VO”. Substantial work was carried out testing the ideas included in this document and helping pipeline teams with the implementation of the recommendations and modifying the recommendations based on feedback from the pipeline implementations.
Given this background, the design principles for the SVO archive were worked out, and it was determined that a highly normalised data-base structure should be used, in order to minimise the maintenance work and the potential for inconsistencies in the data set. Also, it was decided to implement one database per instrument/data set, and a subset of available metadata for each data set was selected for inclusion in the data base. On top of this structure, it was determined to have a database containing descriptions of each subset. Another design choice was to use a RESTful API to access the data base, upon which it would be easy for other applications to be built (also by third party providers).
The data base implementation was made in PostgreSQL. On top of that (and the RESTful access layer), three clients to access the contents of the data base were made: One web application, one IDL API, and one python API.
Subsets of data sets (AIA, EIT, HMI, SWAP, and Themis) were earlier in the database. In this period, further ground-based datasets have been added to the prototype database (Chrotel, GRIS, IBIS, ROSA). The prototype SVO is maintained by ROB and can be studied at http://solarnet.oma.be/.
Another piece of work that was not immediately included in the prototype SVO but will be useful for all coming SVO is the addressing of the problem of listing events. A VO user is likely to search for a specific type of event or features on the Sun, but defining and cataloguing events is not a trivial matter. Based on capabilities developed under another FP7 project, HELIO (Heliophysics Integrated Observatory; Grant No. 238969), we have tried to enhance the criteria used to select data that are based of the occurrence of an event or phenomena, or the presence of a feature. HELIO is based on a service-oriented architecture where the required capabilities are implemented as independent services; there are many instances of most services with a Registry describing their access. This style of architecture means that the SOLARNET SVO can pick and choose which capabilities it wishes to use.
On the data side, we have enhanced the data search capabilities of HELIO to provide access to data sets that complement those that can be addressed through the SOLARNET SVO. HELIO was able to access a wide range of instruments though a number of data sources. In expanding the capability we have concentrated on trying to provide near continuous coverage in observations from ground-based observatories in wavelengths that are not (currently) observed from space.
We have provided integrated access to the synoptic networks established as part of the GONG programme: magnetograms and white light images are gathered as part of the helioseismology observations and since 2010 observations have also been made in H-alpha. Since there are six GONG sites spaced around the globe, under good conditions these ought to provide continuous coverage, however, bad weather and scheduled down time can result in gaps. To supplement the coverage in H-alpha provided by GONG, we have added other observatories where the observations are extensive and accessible. Such sites include Kanzelhöhe (Austria), Catania (Italy), HASTA (Argentina) and Big Bear (California, US) and we continue to look for other sites. We have also been trying to improve access to image data made at radio wavelengths. Some progress has been made but coverage is still patchy because.
WP60: Advanced Instrumentation Development
This work-package aims at the development of new instrumentation for existing solar telescopes, enhancing their capabilities and scientific potential. Some of the novel concepts and designs explored in this WP may be applicable to future large-aperture telescopes, as well.
sWP60.1: Large Diameter Etalon development
The purpose of task sWP60.1 is to build a prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI), with minimal cavity defects. A preliminary analysis of the scientific requirements for an FPI-based, imaging spectro-polarimeter, to be used with a 4-meter class solar telescope, identified cavity defects of λ/150 @ 633 nm as a necessary property. The work done in this sWP represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
To date, most of the capacity-controlled, tunable FPIs used in solar physics are produced by IC Optical Systems Ltd (ICOS), with diameters between 50 and 150 mm. After several discussions with the company regarding the possibility to improve their default model and the plausibility of reaching the necessary plate flatness over a large diameter, ICOS was selected for the construction of the SOLARNET prototype. Part of the decision rested also on the fact that the INAF partner of sWP60.1 already owned a CS100 controller (an item of high cost), needed to operate ICOS FPIs.
Besides the intrinsic quality of the optical surfaces, common causes of significant cavity defects for the ICOS etalons include gravity, surface coating, preload stresses, and effects of the piezoelectric actuators used in the capacity-controlled system that regulates the spacing between the two etalon’s plates. The final goal of sWP60.1 was to correctly identify and characterize these defects, as well as to propose a path forward to their minimization. After several iterations with ICOS, a modification of the design was agreed upon in the early stages of the SOLARNET project, leading to a fully symmetric design, with two identical plates. A Finite Element Analysis (FEA) of such design was conducted early in the SOLARNET project, and indicated the possibility to further reduce the cavity defects by using the etalon in a horizontal, rather than vertical, configuration.
The prototype was commissioned to ICOS in spring 2015, with a new and improved design that aims at minimizing pre-operation stresses, due to gravity, pre-load etc. The fabrication of such a large, and delicate piece of equipment is extremely complex, and several issues introduced delays in the expected delivery time. These issues included multiple attempts at optically contacting the wedges, and repeated iterations of the plate polishing, to achieve the required figure. By early September 2016, a conversation with ICOS outlined that the plate figure was still not fully to specification. However, given the imminent closing of the whole SOLARNET project, the delivery of the New Prototype ET150 was concerted “as is”, in order to start testing its actual performances, including stability, repeatability, effects of gravity, etc., with the understanding that the cavity would be less than optimal. A procedure for measuring the cavity defects of the etalon had been already developed at the Optical Measurements and Testing Laboratory CNR – INO, and tested on an older ICOS 50 mm etalon (ET50), property of INAF.
An extensive set of tests and measures of the performance of the ICOS ET150 has been carried out. The tests have been performed at the CNR-INO (Istituto Nazionale di Ottica) Lab in Florence (Italy) using a 6” Zygo GPI XP interferometer and a measuring procedure improved at the same lab. The measurement procedure has characterized the ET150, both as a static and as a dynamic system. The tests were centered on the cavity properties: (i) Cavity shape; (ii) Effects of gravity (horizontal vs vertical configuration); (iii) Cavity shape throughout a full range scan
The setup used to measure these properties and the corresponding results are detailed in the corresponding report, revealing strengths and weaknesses of the prototype. The tests show that changing between horizontal and vertical orientation does not distort the cavity significantly, suggesting that the design is robust. The positioning of the piezoelectric actuators does not seem to have an important effect on the cavity shape, either. Future work should focus on improving the surface flatness and the polishing procedure.
The results point towards the feasibility of a 150 mm, capacity controlled FPI built using “conventional” methods, for use in solar imaging spectroscopy. Remaining sources of concerns are whether the desired flatness (λ/150 at 632.8 nm) can be reached with a more careful polishing procedure, as well as the possibility to effectively reduce or eliminate the residual tilts introduced by the closed capacity-controlled loop. At the end of March 2017, the new ET150 has been returned back to ICOS for final polishing of the etalon plates. Further tests will be performed once the prototype is returned from the manufacturing company.
sWP60.2 Image Slicer for 2D Spectroscopy
Integral Field Spectroscopy (IFS) is a technique that provides the spectra of all the points in a bidimensional field of view at the same time and under the same conditions using an Integral Field Unit (IFU). Its application offers information along three dimensions: X, Y, λ. There exist different alternatives of Integral Field Units (optical fibers, microlenses and image slicers). In this sWP the latter option is explored with a prototype construction and testing. Task sWP60.3 has developed the microlenses alternative (see below).
The objective of sWP60.2 was to develop a prototype of an IFU based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope, to make solar observations obtaining the spectra of a 2-D region simultaneously and to validate the concept for the European Solar Telescope instrumentation.
An image slicer is an optical system placed at a focal plane to divide the image into slices which are later distributed generating one or more long-slits. This generated slit feeds a spectrograph as in the case of conventional spectrographs but, since this slit is a reorganization of a bidimensional region, the spectra of all the points of that 2-D field of view are obtained simultaneously.
Image slicers use arrays of mirrors (for pure reflective image slicers) to reflect each part of the image in a different direction using different orientations (tilt X, tilt Y). These orientations are carefully controlled to distribute the images of each ‘sliced’ part of the field of view into the output slit or slits. Thus, the optical path of each ‘sliced’ part of the field of view is defined by the reflection using one mirror of each array and generating a piece of the generated output slit, which is known as ‘mini-slit’. The design of the image slicer for GREGOR was presented early in the project schedule and later updated, after including modifications required by manufacturing tolerances. The prototype includes not only the image slicer but also the field-of-view scanning system (FoV-SS), optical interface to couple it to GRIS, and the optical bench that holds the system. The image slicer was constructed and finally received in October 2016.
The last period of the project was dedicated to finalize the construction of all opto-mechanical pieces and to test the full unit at the lab and telescope, once the IFU was received. The full system was installed at the telescope in November 2016. The setup used at the lab, the tests at the telescope and the results are described in a SOLARNET document. The tests were devoted to determine the mechanical and optical performance of the whole system: Optical and mechanical alignment of FoV-SS and slicer, scanning system accuracy and repeatability, image quality of the slicer, cross-talk, and stray-light. After all these tests, the successful performance of the whole system was demonstrated. New tests at GREGOR are scheduled for the end of June 2017, together with scientific observations.
sWP60.3 Microlens-fed Spectrograph
Task sWP60.3 addresses the design and construction of a microlens-fed spectrograph to measure simultaneously all points of a given area on the solar surface a given area, as an alternative to the Integral Field Unit based on image slicers described above. A prototype, called MiHI (Microlensed Hyperspectral Imager), has been designed and constructed.
The MiHI is a prototype of a microlens-fed hyperspectral imager, specifically designed for solar observations, and intended to overcome the limitations of traditional solar instrumentation. It simultaneously resolves the spatial and spectral dimensions, thus promising data with a S/N ratio close to the theoretical maximum. The challenges of developing such an instrument for solar applications differ from those for night-time applications, both in terms of the necessary control of the amount of stray-light produced by the spatial extent of the object, as well as the spectral resolution, that must be high enough to resolve individual atomic lines in the solar spectrum with sufficient spectral and spatial resolution to derive the physical conditions in the solar atmosphere in 3 spatial dimensions, as well as in time.
The MiHI solves these challenges by using a specially designed and manufactured microlens array that allows all resolution targets to be achieved simultaneously, with a minimum of experimental elements, while retaining an acceptable instrument transmission. The design considerations can be found in the corresponding SOLARNET document.
The principle of operation of a hyperspectral imager is typically based around a device that reformats the image plane, thus making space for the spectral dimension. The MiHI uses a double sided microlens array to reduce the size of each of its image elements and accomplish this goal. To avoid overlap of the spectra, an ultra-narrow bandpass filter is required. The filter must be as high-order as possible, to minimize the contamination from one image element to the next, and to optimize the signal level over the maximum possible spectral range. Steepness of the flanks of the filter transmission profile, as well as maximum peak transmission is therefore of the utmost importance.
To avoid designing and building a spectrograph in addition to designing and building the image formatter, the MiHI prototype was designed as a drop-in replacement for the slit-box of the existing TRIPPEL spectrograph, currently installed at the Swedish Solar Telescope (SST). A number of tests haven carried during an observing campaign carried out in October 2016 at the Swedish Solar Telescope (SST) with the setup described above. The prototype was installed in front of the TRIPPEL spectrograph, aligned and adjusted, after which some of the basic properties were characterized. All opto-mechanical elements required for the installation of MiHI at the STT were constructed during the third reporting period. Their description and alignment procedures can be found in the corresponding report.
The instrument was found to generously exceed the spectral resolution requirements, and meet the spatial resolution and field-of-view (FOV) requirements, but is somewhat less transparent than intended (25%, where 50% was the target), and shows some signs of minor thermal drifts. The reduction and calibration of the data was found to be challenging, and will require considerable further modelling and observing experience to produce high quality data. All details can be found in the SOLARNET report.
sWP60.4 Fast Imaging Polarimeter
The goal of the Fast Solar Polarimeter (FSP) project was the development of a novel solar imaging polarimeter with an emphasis on significantly increased polarimetric accuracy and high spatial resolution. The instrument is based on a fast polarization modulator synchronized to a high frame-rate and low-noise detector, working at a polarization modulation frequency in the 100 Hz range. This increased modulation frequency suppresses spurious polarization signals induced by external disturbances such as atmospheric turbulence or telescope jitter and avoids crosstalk between polarization states, in particular between linear and circular polarization, which limits the accuracy of today’s slower dual-beam polarimeters. In addition, the fast modulation allows to obtain time series of numerically restored Stokes images at a cadence below the typical solar evolution time scale of magnetic structures at the smallest resolved scales (of the order of 100 km on the Sun, corresponding to evolution time scales in the range 1-10s). The higher polarimetric accuracy that can be achieved with FSP will open up new windows for studying solar magnetic fields and is, in particular, expected to advance studies of the enigmatic magnetic field in the quiet Sun and of chromospheric magnetic fields. The instrument is particularly developed in preparation for the next generation of large aperture solar telescopes like the planned European Solar Telescope (EST).
The FSP modulator and its performance in single-beam configuration have been tested and described in the corresponding report. Another document has been produced in which the polarizing beam splitter (PBS) for the dual-beam configuration is described, together with the test results demonstrating its performance. This high-precision optical device has been invented and designed by MPS, in collaboration with the Canadian optics manufacturer LightMachinery Inc. The PBS was delivered to MPS in December 2016.
The optical and polarimetric performance of the PBS has been tested and compared to the specifications. The setups used to measure these properties and the corresponding results are detailed in the corresponding SOLARNET deliverable. The first scientific application will be at an observing campaign in August 2017 at the IBIS filtergraph instrument of the Dunn Solar Telescope (DST) in New Mexico, USA.
WP70: Wavefront control: turbulence characterization and correction (led by IAC)
WP70 is divided into 3 sWPs, with a total of 5 tasks, two related with adaptive optics issues (sWP70.1) one related to seeing measurements (sWP70.2) and the other two related to seeing effects minimization (sWP70.3):
sWP70.1. Adaptive Optics (AO)
Task sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
Task sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
sWP70.2. Atmospheric Seeing Characterization
sWP70.3. Local Seeing
Task sWP70.3.1 Application of CFD techniques for local seeing optimization
Task sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
A particularity of solar AO is that it uses the solar granulation as a reference; therefore the wavefront sensing is performed using correlations on images with a field of view of about 10”. A sensor collecting such a wide field of view averages wavefront information from different sky directions, affecting the sensing of high altitude turbulence, the sampling of which does not depend anymore on the size of the subapertures only, but rather on the size of the projection of the extended field of view. Understanding this effect is crucial for the design of future solar facilities, i.e. to choose the adequate height of the DMs on MCAO systems, and also to predict the quality of the reconstruction that such system would be able to achieve. For that reason, we have studied wide field sensing and explained the analytical equations that describe the process. The equations have been validated with the results of the numerical simulations run with the code FrIM. A complete error budget has been evaluated, including the generalized fitting error.
The influence of the correction order of turbulent layers in an MCAO system in the visible has been simulated using the physical propagation approach. Static phase screens have been generated following a Kolmogorov statistics. The outcome of this study is that amplitude propagation errors can be neglected, while phase propagation errors due to non-linear effects have to be considered. A perfect cancellation of phase errors is achieved if the MCAO correction is done in the inverse order of turbulence occurrence. Nevertheless, the degradation of the performance is not critical if high turbulent layers are corrected in the order of occurrence.
Specific simulations have been performed under different turbulent conditions to determine the optimum height for the MCAO DMs. Using real atmospheric profiles from ORM and OT, we show that, after correction, a homogeneous Strehl higher than 40% can be achieved over a 1 arcmin FoV for all elevations. These numerical results are validated with the analytical equations and the complete error budget commented above.
For the determination of the optimum conjugated heights of the MCAO DMs, a method to characterize the day time turbulence up to several tens of km is derived. The SHABAR instrument used to determine the diurnal seeing conditions (see sWP70.2) is only sensitive up to a distance of 2-3 km to the turbulent layer. The contribution to the seeing of higher layers is crucial to understand the image degradation and, consequently, other means are required to explore the behavior of these high-altitude layers. To complement the results obtained with the SHABAR, we make use of the instrument SCIDAR, installed at the Observatorio del Teide (OT) and which can derive turbulence profiles during the night. We have also used data taken by regular radiosondes that are launched in Tenerife (twice per day). The three data sets (SHABAR, SCIDAR and radiosonde) have been adequately compared and merged to produce a single turbulence profile which may be considered as typical of the atmospheric conditions at the OT. This profile has been used for the optimization of the MCAO system and presented in a report.
Efforts were also dedicated to forecast the MCAO correction. To that aim, a hardware demonstrator was constructed to test our forecasting algorithm FORS (closed loop FORcasting System). We studied the outcomes of introducing both a simple periodic defocus aberration and a real open loop defocus time sequence acquired at the VTT solar telescope. In both cases, FORS grants a significant performance increase, improving the stability of the system in closed loop conditions, and decreasing the amplitude of the residual uncorrected wavefront aberrations.
sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
AO systems are installed at most existing solar telescopes to improve their image quality, with excellent results in imaging observing modes, but the combination of polarimetric measurements with adaptive optics is extremely challenging and it is not usually fully addressed. Being THEMIS a solar telescope which provides high polarimetric performance, the implementation of an AO prototype to improve its image quality retaining its unique spectropolarimetric capabilities, would open the possibility to specialize this telescope in high resolution in polarimetry. The possibility of performing very high quality polarimetric measurements using AO is one of the challenges of the future large aperture European Solar Telescope (EST) and the implementation of an AO system at THEMIS will provide an excellent bench to test these observing techniques for EST.
A full study has been conducted to implement an adaptive optics system for the THEMIS telescope, taking into account the current technical state of the telescope and the scientific goals derived from the polarimetric usage of the long slit spectrograph attached to the telescope. It shall be clear from the beginning that THEMIS has been specifically designed to be a "calibration-free" polarimetric telescope and that the AO design has been done keeping this important specification valid. The adaptive optics (AO) prototype for THEMIS is composed of two separate and complementary branches. On the one hand, there is the design and construction of the AO optical bench itself, including the wavefront sensor, the pupil deformable mirror and all the accompanying mechanical and optical parts. On the other hand, some changes in the mechanics and optics of the telescope are required to accommodate the adaptive optics system.
During the initial phases of the project, the design of the AO prototype was accomplished and the corresponding reports and deliverables were produced. Later, the detailed design of all changes required at telescope level was produced and providers of all new optical and mechanical elements were contacted and purchase orders submitted. Finally, all these elements have been received and tested at the laboratory during the third reporting period. The design is such that the same optical bench can be moved from the lab to the telescope as it is. New mirrors M2 to M5 of the telescope have been acquired, tested and installed at the telescope with their corresponding mounts. The performance of these new optomechanical parts has proved to be within specifications. Global on-sky testing of the whole system is planned for fall 2017. The adequate reports have been generated where all these achievements are presented.
sWP70.2 Atmospheric Seeing Characterization
Two long-baseline SHadow BAnd Rangers (SHABAR) instruments were deployed by the IAC in the Canary Islands during 2010, one at Observatorio del Teide (OT), in Tenerife island, and another at Observatorio del Roque de los Muchachos (ORM), in La Palma island. The instruments acquire sunlight signal data that can be reduced to produce Cn2 and r0 profiles for the lower atmosphere layers up to some 2-3 Km height. The mission of these instruments, together with other daytime turbulence measurement instruments, is the characterization of the daytime sky in both sites, OT and ORM, in order to select the best location for the European Solar Telescope.
Reports have been presented in which the data acquisition and analysis are described, as well as the main results of the data analysis performed showing the statistics and seeing properties at different heights from the ground and the daily variation for the different seasons of the year. These results are presented for both Canarian observatories: OT and ORM. In the figure below, the variation of the median r0 with height at both observatories is presented. More details can be found in the corresponding document, including the diurnal and seasonal variation.
sWP70.3.1 Application of CFD techniques to the local seeing optimization
We have developied numerical methods to estimate the atmospheric seeing and simulate some test cases of certain configurations of the EST. The main feature of the proposed model is that it is based on the numerical dissipations which arise from a particular version of the Variational Multiscale Method, the Orthogonal Subgrid Scale method. It is a finite element method where one assumes that the solution can be split into large and small scales. The former are considered those captured by the finite element mesh, whereas the latter are considered L2-orthogonal to the former and modeled approximately. The advantage of using this kind of models lies on the fact that, by decomposing the fields of interest into coarse and fine scales, they are able to deal simultaneously with the sources of numerical instabilities and the modeling of turbulent effects.
We have tested the performance of the method in three practical cases, namely a convective boundary layer, the flow inside a transfer optics room, and the flow around a telescope enclosure. In all three cases we have compared our model with the results obtained by using a Smagorinsky and WALE models for evaluating the viscous and thermal dissipations, and, in the convective boundary layer case, with the results presented in the literature. The numerical examples show that the method is capable of doing an accurate estimation of the Cn2 coefficients. This fact does not only provide us with a new numerical tool for the evaluation of the atmospheric seeing but it also adds arguments in favor of the viability of implicit LES methods which rely on the numerical stabilization mechanisms for the modeling of turbulence.
In a second stage, efforts were concentrated on completing the set of CFD simulations to evaluate the seeing conditions in several configurations of the telescope facilities for EST and help the SOLARNET consortium to decide the optimum design of the EST facilities. To that aim, different configurations were analysed for the telescope facilities with a conventional dome (closed configuration) and facilities with windshield (open configuration), in summer, for North wind of 5 m/s, in order to select the optimal configuration. The analysis include different moments of the day, since the ambient temperature, the ground temperature and the temperature of the facilities changes along the day and, hence, the seeing degradation is different. Also, several conditions on thermal control have been taken into account to obtain an estimate of the convenience and effect of the thermal control mechanism. The final objective of the analysis was to keep the temperature of the surfaces of the facility as close as possible to the ambient temperature so as to minimize the local seeing effect. Once the temperature maps were obtained for the selected configurations, the CFD analyses were performed to evaluate the local seeing degradation.
Finally, an analysis of different alternatives for the EST telescope structure has been performed. The analyses have been centred on the Rocking-Chair, Gantry and Yoke models. The performance of each structural model has been evaluated according to their dynamic behaviour, as well as to the errors induced by gravity and the wind loads acting on the structure. An alternative configuration for the upper section of the telescope tube has also been considered, as well as the possibility of changing the current Nasmyth platform position from the right side of the telescope to the rear side.
All the analyses, simulations and the results are described in the corresponding reports.
sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
Task 70.3.2 is devoted to the design and construction of a heat rejecter (HR) prototype for the GREGOR telescope. This HR follows the concept of the design proposed for EST, and, consequently, represents a proof of concept for the technical solution.
A thermal analysis was done to study the current contact sensors layout, evaluate the thermal environment at GREGOR F1 and estimate the possible improvement in the reduction of local seeing effects. A “cold” prototype was constructed to check the leakage, and the hydraulic and thermal performance. The Heat Rejecter (HR) final prototype has been produced and is ready for delivery to the GREGOR telescope. The HR front surface has been polished and coated. The device includes three interchangeable field diaphragms. Laboratory tests have demonstrated the good performance of the system. The final prototype has been realized after modifications deduced from the experience achieved with the first “cold” prototype to reduce the hydraulic friction resistance inside the HR body and pipes. A number of hydraulic tests have been performed on the final prototype. The heat rejecter will be integrated and tested at the telescope during the next maintenance period of the GREGOR telescope (foreseen for summer 2017).
WP80. Synoptic observations: Solar Physics Research Integrated Network Group (SPRING)
A network of telescopes with a small aperture but a large field-of-view can provide useful data to prepare observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures.
During the first period of the project, the scientific requirements were defined, from which the observational requirements were derived and translated into technical requirements. The translation into technical requirements followed the key idea to obtain a simple observatory, which is easy to operate and most cost effective. From the list of the observational requirements it is clear that a single instrument is not capable of fulfilling all those needs. Even if it can, its technical complexity would be too high, which finally would lead to high costs and low mean time between failures. For a network that should provide the data without interruption, this is not acceptable. Therefore the development had to follow the concept of a single platform that host several instruments.
Starting from the requirements for the spatial resolution, the detectors, instruments and front-end telescopes were defined. The technical feasibility of the proposed concept was extensively tested using the guiding telescope of the Vacuum Tower Telescope (VTT). Instead of a prefilter wheel a fast matrix filter shifter was used to obtain full-disk Dopplergrams in various wavelengths. Dual Fabry-Perot interferometers were operated in a collimated mount.
The overall technical concept is described in the corresponding document, including a study on the seeing conditions at the six possible sites Teide Observatory, Big Bear Observatory, Cero Tololo, Mauna Loa, Learmonth and Udaipur, estimates for data rates, and a description of the cloud monitoring.
In order to obtain a cost estimate, the combination of cameras, instruments and telescopes needed to be combined. Furthermore the cost for the building and the telescope tower were taken into consideration. The two concepts one with the fulldisk telescope (FDIS) and one with the multiplexed spectrograph (mxSPEC) result in a cost of up to 51.3 Mio. Euro for setting up a network with six nodes around the world.
Overall, the study was successfully completed. SPRING has gained large attention by the scientific community. The need for synoptic observations of the Sun is expressed by the world-wide research community. Plans for setting up a ground-based network of observatories can now be built on this technical and feasibility study.
WP90. Access Programme to key ground-based infrastructures (led by KIS)
The oversight of the TAS programme is described in the report on WP20. The ACCESS time at the SOLARNET telescopes and instruments was allocated and delivered successfully. The ACCESS programme enabled a large number of users to take advantage of state-of-the-art facilities. In particular it brought a large number of new users to the telescopes. It helped to foster small national European communities which cannot afford such large facilities.
All relevant number on the ACCESS programme are given in the document "Report on the TAS programme 2013 - 2016". A total of 466 observing days were allocated and executed. These are 15 days more than planned, but since the real unit costs turned to be slightly smaller than the initial estimated costs, the used budget amounts to almost the same amount.
Both PI and service mode campaigns have delivered valuable state-of-the-art data sets that led to scientific publications in refereed scientific journals.
WP100. Access to Science Data Centres. Space missions
The centers have all streamlined the access to the facilities and have significantly added new resources to the databases through the SOLARNET project. The three centers have increased awareness of the existence of the space-based data to the solar physics community in Europe. This has been accomplished through participation in conferences and meetings. More details about the individual centers are given below.
SubWorkpackage 101: HINODE/IRIS. Access to Science Data Centres (UiO)
Detailed monthly reports on the activity in the Science Data Centre have been produced followed by detailed usage statistics (complete data available from http://sdc.uio.no/sdc/webstats). Access to the Hinode Science Data Centre was provided throughout the whole project. The most significant improvement through the SOLARNET project was the addition of all data from the Interface Region Imaging Spectrograph (IRIS). The data was made available from the first day of open data policy, October 31st 2013. During the reporting period, all data from Hinode and IRIS have been continuously ingested into the Data Centre. Additional search possibilities have been added and the data presentation has been further improved through improved thumbnail pictures.
SubWorkpackage 102: BE-WISSDOM. Access to Science Data Centres (ROB)
The Solar Dynamics Observatory (SDO) was launched in January 2010 carrying three primary instruments the Atmospheric Imaging Assembly (AIA), the Helioseismic and Magnetic Imager (HMI) and the Extreme ultraviolet Variability Experiment (EVE). These instruments produce between 1 and 2 TB of data per day. The goal of the European SDO data centre is to be a powerful cache for the data produced by SDO. By doing this we diminish the load on the USA SDO data centre at the Joint Science Operations Center (JSOC) at Stanford university and provide the Solar community with a quick and reliable access to SDO data.
The hardware of the SDO data centre was updated in 2015. The SDO datacenter now has the following hardware: The storage array is made up of a SuperMicro storage server and a JBOD (just a bunch of disks) storage extension. These are equipped with 45 4TB hard disks and two 80 GB solid state drives, providing about 120 TB of usable storage after formatting. The database server was purchased with 80 GB and 800 GB SSD hard disks to store the indices of the databases, maximum of 12 SATA/SAS hot-swap hard disks, a redundant power supply and 256 GB double data rate type SDRAM.
SubWorkpackage 103: GSC-SDO. Access to Science Data Centres (MPS)
The German Science Center for the Solar Dynamics Observatory (GSC-SDO) is dedicated to the acquisition and distribution of data from the Helioseismic and Magnetic Imager (HMI) instrument that is onboard the Solar Dynamics Observatory (SDO). The GSC-SDO is hosted by the Max Planck Institute for Solar System Research (MPS) in Goettingen, Germany.
The ultimate goal of sWP103 is to make HMI data (and other relevant data) available to the European solar physics community through the Data Record Management System, NetDRMS. NetDRMS is a software suite developed by Stanford University specifically for managing SDO observations and sharing the data worldwide: http://jsoc.stanford.edu/netdrms . NetDRMS is fully operational at the GSC-SDO and data are automatically transferred directly from Stanford University as soon as they are available. The GSC-SDO has an online raw storage capacity of 888 TB dedicated to SDO/HMI data and other related data products.
The GSC-SDO is “subscribed" to the several data series from the HMI instrument. When a NetDRMS site subscribes to a particular data series, that site by default receives all the meta-data, i.e. keyword, value pairs, directly from Stanford University. There is a separate mechanism that must be triggered in order to also receive the images. Furthermore, this can be done selectively so you can choose which images you want in your local NetDRMS system. This latter feature is particularly useful if local storage resources are not sufficient to store all the images, allowing a site to only transfer the images of interest.
The MPS strongly promotes the GSC-SDO whenever possible. In particular, it is described in many posters from the MPS that are prepared for meetings/conferences worldwide. Many MPS publications also acknowledge and/or describe the GSC-SDO. This is a consequence of the GSC-SDO being a local facility for MPS scientists, and hence many of the scientific results coming from the MPS made use of data from the GSC-SDO.
Potential Impact:
IMPACT, DISSEMINATION ACTIVITIES AND EXPLOITATION RESULTS
The SOLARNET project has represented a major initiative for the field of high-resolution solar physics. The objectives of the project have been successfully achieved with a considerable impact in all areas aimed at the beginning. In the following paragraphs, this impact of SOLARNET in the most relevant areas is briefly described.
1.- Integration of the key research infrastructures
The most prestigious and powerful European facilities in the field have been offered to the whole community in a co-ordinated way. SOLARNET has organised the access to the German VTT and GREGOR, the Swedish SST, and the French THEMIS telescopes, as well as to the Italian IBIS and British ROSA instruments operating at the American DST telescope. This access has been managed by the EAST Time Allocation Committee (TAC), composed by one representative of the institutions that operate the telescopes/instruments handled by SOLARNET, plus one representative of IAC and NSO.
The EAST TAC granted observing time in PI mode as well as in the ‘service’ mode. The PI mode is the traditional observing mode, in which the principal investigator and parts of his/her team goes to the telescope and performs the observations. The technical support is typically limited to the telescope and the instruments need to be operated by the observing team. The service mode is a novel observing mode in solar physics and it is fundamental for an optimum operation of future facilities. In this mode the proposers describe the scientific objective and give a detailed explanation on what data they need. The data is acquired by a service team at the telescope following the requirements that the proposers have specified.
The EAST TAC announced calls for proposals for the observing seasons covered by the SOLARNET period. The calls were issued in ‘SOLAR NEWS’, which is subscribed by most Solar Physicists worldwide. Also, the EAST TAC informed all ‘known’ people that could be interested in the ACCESS program. In addition the Project office implemented a web page within the SOLARNET internet portal to promote the ACCESS program, and distributed a poster on the ACCESS program. The proposal deadlines were in mid-January of each observing season. The number of submitted proposal was higher than expected. The EAST TAC checked the proposals for eligibility and contacted the telescope operators for checking the technical feasibility. The EAST TAC selected two independent referees who rated the proposals according to their scientific merit. Based on the scientific merit, the EAST TAC granted the observing time.
The ACCESS programme was most successful in being oversubscribed and bringing new users to the telescopes. In the course of the 4-year program, the oversubscription rate increased steadily. In 2016 - the fourth year - the ACCESS program was oversubscribed by 200% (26 proposals for 13 allocated projects). If only PI mode campaigns are considered, the oversubscription rate was even higher. It amounted to 250% (22 proposals for 9 allocated projects).
During the whole project, the EAST TAC allocated a total of 68 observing campaigns with 466 observing days. 353 users from 18 countries were awarded with observing time and 59 received a travel & subsistence grant to perform the observations. 30 of those 59 were new users.
SOLARNET brought together all the data pipeline experts from the SOLARNET telescopes, as well as US data scientists that are involved in the data pipelines of the upcoming 4m solar telescope DKIST in Hawaii. Having brought together this community is a major success of the project. A main result of this joint activity is the generation of data pipelines for the major European solar instruments. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories. SOLARNET issued guidelines with general recommendations for those pipelines, so that external observers could easily get their data reduced to facilitate the posterior analysis. Thanks to SOLARNET, there are now modern pipelines for all major European solar instruments. The pipelines have been raised to a new standard where they are more reliable and easy to use. This has led to a higher productivity and more and better science. The work with conforming the pipelines to the needs of virtual observatories will allow the future development of such databases.
A number of prototypes have been designed, constructed and tested to improve the future capabilities of the facilities handled by SOLARNET, and having in mind the future large aperture European Solar Telescope (EST):
a.- A full study has been conducted to implement an adaptive optics system for the THEMIS telescope, taking into account the current technical state of the telescope and the scientific goals derived from the polarimetric usage of the long slit spectrograph attached to the telescope. The AO design has been done keeping valid the important specification property of this telescope as a "calibration-free" telescope.
b.- An innovative heat rejecter prototype for GREGOR has been designed and fabricated. This heat rejecter follows the concept of the design proposed for EST, and, consequently, represents a proof of concept for the technical solution proposed for this new large facility.
c.- A prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI) has been built. The results point towards the feasibility of a 150 mm, capacity controlled FPI built using “conventional” methods, for use in solar imaging spectroscopy. This activity represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
d.- Two prototypes of integral field units to obtain the spectra of all points in a 2-D region simultaneously have been especially designed and constructed for the telescopes GREGOR and SST. One is based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope. The other is based on using microlenses to feed the spectrograph TRIPPEL of the SST. Both prototypes have been successfully tested at the corresponding telescopes and validate these concepts for the European Solar Telescope instrumentation.
e.- A fast solar polarimeter has been designed and tested using novel detector concepts, especially designed within SOLARNET, and a new polarizing beamsplitter.
2.- Contribution to a Solar Virtual Observatory
The standard for metadata and data archiving have been defined and adopted by the various SOLARNET telescopes and instruments. In the community this is known as the SOLARNET standard for metadata and is most relevant to the community world-wide. The team of the US 4m solar telescope DKIST who is designing the concept for their data pipelines is also taking advantage of 'our' European SOLARNET standard. An SVO prototype has been produced that allows the retrieval of ground-based and space-based data. A data archive exists for the GRIS instrument and science ready can be retrieved by external users. The GRIS instrument data pipelines provide the necessary metadata to be ingested into the SVO prototype, such that the GRIS data is fully searchable by external scientists. A similar data archive is provided by BBI@GREGOR, GFPI@GREGOR, and the Italian IBIS@DST data. Their metadata is also ingested into the SVO prototype. The prototype SVO can be accessed at http://solarnet.oma.be/.
3.- Contribution to the realisation of a future large European ground-based facility (EST)
All the aspects mentioned above (access to external observers, service mode, pipeline development, guides for data formats to accommodate virtual observatory rules, SVO prototype fed with high-resolution ground-based data, improvement of telescope capabilities and new concepts of instrumentation) are key aspects in which the impact of SOLARNET on the major European high-resolution facilities to observe the Sun has been made apparent. They are also crucial for the definition and operation of the European Solar Telescope (EST), so that this facility is optimally designed according to its scientific goals, operated in an efficient way and delivers data that is accessible to the whole community.
In addition to those achievements, specific works have been carried to optimise the design of EST. Specifically, SOLARNET has addressed the following tasks:
a.- A site-testing campaign has been carried put to characterise the turbulence above the two Canarian observatories (Observatorio del Teide, in Tenerife, and Observatorio del Roque de los Muchachos, in La Palma). Results have been produced showing the statistics and seeing properties at different heights from the ground and the daily variation for the different seasons of the year. This SOLARNET activity will be most important for the final decision of the location of EST.
b.- Specific simulations have been performed under different turbulent conditions to determine the optimum height for the multi-conjugate adaptive optics system (MCAO) deformable mirrors (DM). Using real atmospheric profiles from ORM and OT and the evaluated MCAO DMs, it has been shown that, after correction, a homogeneous Strehl higher than 40% can be achieved over a 1 arcmin FoV for all elevations. These numerical results have been validated with analytical equations and a complete error budget.
c.- Thermal analyses of the EST environment have been done for different configurations and weather conditions. These analyses have been used for specific CFD simulations to evaluate the seeing conditions and help the SOLARNET consortium to decide the optimum design of the EST facilities.
d.- A study of different alternatives for the EST telescope structure has been performed. The performance of each structural model has been evaluated according to their dynamic behaviour, as well as to the errors induced by gravity and the wind loads acting on the structure.
e.- SPRING, a network of telescopes with a small aperture but a large field-of-view has been defined and the technical solutions studied. This network is useful to provide data that can help the preparation of observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures.
4.- Synergies with other communities
Four meetings were organised by SOLARNET to foster synergies between our high-resolution ground-based solar physics community with other fields of research. Specifically, these meetings addressed the topics: “Synergies between ground- and space-based solar research”, “Solar and stellar magnetic activity“, “The Sun, the stars and solar-stellar relations”, and “The physics of the Sun from the interior to the outer atmosphere”. The increasing impact of SOLARNET within the community is demonstrated by the fact that an increasing number of attendees participated in the SOLARNET meetings (60, 61, 84, and 143, respectively).
5.- Strengthening and reinforcement of the European Research Area
Modern science is entirely global, with information shared between researchers across the world virtually instantly. For astronomy and other similar disciplines there is another reason for globalisation - the high cost of the infrastructure that we need today. This means that countries must work together to plan for the future, pooling available resources across the whole of Europe.
SOLARNET has led to an increased mobilization and movement of human resources across Europe and their exposure to different ideas and methods, which are likely to lead to an improvement in career development prospects, particularly for the young researchers involved in the project. The adequate training of young researchers is fundamental is most important to create the atmosphere for international co-operation, spread knowledge and reinforce the idea of a European Research Area.
With that goal in mind, SOLARNET has carried out an ambitious Mobility Programme with seven calls for young researchers to allow them to begin early to establish international collaborations and reinforce the contacts between different groups. 20 young researchers benefitted from this programme, with an average stay of 8.3 weeks at another institution.
As a complement of the Mobility Programme, SOLARNET has organised five schools on fundamental research areas of solar physics: “Introduction to Solar Physics”, “Ground- and space-based inseruments”, “Solar magnetic fields: modelling and measuring techniques”, “Solar MHD and reconnection”, and “MHD waves and oscillations in the solar atmosphere”. The schools were accompanied with related Workshops that facilitated the interaction of the young researchers with well-established scientists and the presentation of their research work.
6.- Innovation and socio-economic impact
SOLARNET has contributed to reinforce significantly the relationship with industry for the most advanced technological challenges addressed by the project. This is the case of the partners Winlight Optics (for the fabrication of the imager slicer for 2D spectroscopy), SRS (for the fabrication of the heat rejecter prototype for the telescope GREGOR), PNSensor (for the development of high-speed low-noise new sensor concepts), and HANKOM (for its participation in the minimisation of the impact of local turbulence on image quality).
In addition to this direct participation in the most technical works, innovation aspects were channeled through the participation in the project of TECNALIA. This partner has a wide experience on this type of activities at EU level with industrial experts, and their European networks and contacts, on Technology Transfer. The particular achievements achieved can be summarized:
a.- Identification and analysis of technologies, techniques and new concepts related to solar physics telescopes and instruments
b.- Dissemination of the innovative aspects and findings of the project towards industry
c.- Organisation of workshops related to technology transfer
d.- Assessment of space technologies
e.- Dissemination of the project (specifically the technology offer and needs)
f.- Organisation of Common Exploitation Booster workshop
g.- Identification of future projects
h.- Promotion of synergies with other EU projects
7.- Dissemination activities
The main objectives of SOLARNET for the dissemination and exploitation of the knowledge derived from the project have been:
a.- make solar physics research more attractive and more comprehensible for a targeted audience.
b.- achieve visibility for the project and its findings across Europe and worldwide, both inside and outside the scientific community;
c.- promote the awareness of science.
d.- encourage talented students and scientists to join the partner institutes and enterprises;
e.- generate industrial interest for the developed products or services.
In order to spread this knowledge, the outreach and dissemination approach of the consortium has foreseen a well-defined set of targeted audiences at European level: the scientific community (with the presentation of SOLARNET activities results and the EST project to the main scientific meetings), industry (with the organisation of workshops related to technology transfer and presenting SOLARNET at a number of technological fora), policy makers (SOLARNET and EST have been continuously presented to the national funding agencies), media and general public (press releases and publicly available videos have been produced).
The European Strategic Forum for Research Infrastructures (ESFRI) has recognized the efforts done by the high-resolution Solar Physics community with the SOLARNET project and has included EST in the list of strategic European infrastructures (ESFRI roadmap) in March 2016. This is a major achievement that undoubtedly will foster the community to higher and more ambitious goals to make present telescopes more efficient, increase the scientific return of the data taken with the most advanced facilities, and make EST become a reality.
List of Websites:
http://www.solarnet-east.eu/
Contact details available at http://www.solarnet-east.eu/contact
SOLARNET is a project aiming to bring together and integrate the major European research infrastructures in the field of high-resolution solar physics, in order to promote their coordinated use and development. The project involves all pertinent European research institutions, infrastructures, and data repositories.
Networking activities, access to first-class infrastructures and joint research and development activities have been covered under SOLARNET to improve, in quantity and quality, the service provided by this European community.
SOLARNET involves 32 partners from 16 countries: 24 EU research institutions, 6 EU private companies and 2 USA research institutions. The co-ordinating institution is the Instituto de Astrofísica de Canarias.
SOLARNET provides access to leading world-class ground-based research infrastructures, aims at equipping them with advanced instrumentation, and prepares for the next generation of high-resolution telescopes, with the European Solar Telescope (EST) as the most prominent facility.
The following actions have been carried out under the auspices of SOLARNET:
- Transnational Access to external European users.
- Enhance and spread data acquisition and processing expertise to the community.
- Increase the impact of high-resolution data by offering science-ready data.
- Provide unified access to pertinent data repositories with space and ground-based data.
- Foster synergies between various research communities by organising joint meetings.
- Train a new generation of solar researchers through schools and a mobility programme.
- Develop prototypes for new-generation post-focus instruments.
- Study local and non-local atmospheric turbulence and ways to negate its effects.
- Improve the performance of existing telescopes.
- Improve the design of future large European ground-and space-based solar telescopes.
- Lay foundations for combined use of facilities around the world and in space.
- Foster partnership with industry to promote technology transfer with existing networks.
- Disseminate activities towards society.
The project started on April 1st 2013, and the 3rd reporting period covers since April 1st 2016 to March 31st 2017. The progress achieved in this period is summarised below.
Project Context and Objectives:
WP10: Project Coordination and Management
This work-package has the following main objectives:
-Effective coordination and management of the whole project.
-Implementation of an effective and transparent management of the project, ensuring appropriate control and monitoring: schedule, budget & scope (milestones and deliverables).
-Effective communication among activities and teams.
-Monitoring and reporting of the overall progress and use of the infrastructures for access, assuring the timely delivery of high quality deliverables and milestones.
-Contractual and financial follow-up of the project.
-Coordination of dissemination activities.
In order to guarantee an appropriate management structure and administration of the Project and its tasks and activities, an elaborated management structure was set up
WP20: Integrated operation and exploitation of solar physics facilities and coordination with other research infrastructures
The focus of this work package has been to optimize the use of the existing solar facilities and research structures. To achieve this goal, we have worked on a number of objectives: (20.1) coordination and implementation of the Transnational ACCESS and Service program, (20.2) to make available proper data pipelines for existing instruments, (20.3) development of guidelines for metadata, aiming at a standard for archived solar data, (20.4) coordinated observations of our facilities with non-European and/or non-optical infrastructures, (20.5) study new ways of operating our telescopes, in particular, gather experience with performing observations in service mode.
WP30: Solar Physics Networking
The aim of this work-package has been to foster collaborations among different solar physics groups, promote the interaction and cooperation among researchers of different level of expertise, as well as to encourage and promote synergies with other fields. Some of the actions foreseen under this work-package have been: exploitation of ground- and space-based data; enhancement of collaborations with other communities and Projects; promotion of collaborations between the new generation of scientists and experienced researchers through short stays and training actions to acquire competences in relevant fields of solar physics.
sWP30.1: Meetings on solar physics
One objective of this sWP has been to put in contact different solar physics communities and researchers involved in different fields of research
sWP30.2: Mobility
SOLARNET supported the mobility of young researchers as part of the networking activities in this work-package. The mobility programme has been designed, as an additional aspect of the training program, to reinforce the contacts between different groups and to allow young researchers to begin early to establish international collaborations.
sWP30.3: Training (Schools and Thematic Workshops)
The aim of this task was the organization of summer/winter schools for PhD students and novel post-doc researchers on topics related to the development of new instrumentation for solar observations, diagnostic tools, hot solar research topics and fields of mutual interest for solar and stellar physicists. This task had the goal to organize 5 Schools and Thematic Workshops.
WP40: Innovation Towards Industry
The overall objectives of WP40 can be defined as follows:
-Identification and analysis of technologies, techniques and new concepts already in use at the forefront of solar physics’ developments with potential interest for other sectors.
-Dissemination of the innovative aspects and findings of the project towards industry, through the existing and highly successful industrial associations and related high-tech companies at EU level.
-Assessment of space technologies of most interest for solar physics developments, and identification of common needs and opportunities.
-Extension of the effort of collaboration between high-tech companies, universities and public organisations on solar physics and related fields, oriented to the transfer of knowledge using well proven methodologies.
-Increase of the potential for innovation of research infrastructures on solar physics.
Promotion of synergies with other I3 initiatives on this same topic.
WP50: Tools for innovative data handling: pipelines, databases and SVO (led by SU)
Two were the main goals of this work-package:
(i)Develop data-reduction pipelines for the most important European ground-based high resolution solar instruments. Enhancement of observational procedures for increased productivity and easier co-observing and combination of data. The pipelines will produce data and meta-data fulfilling the requirements of a Solar Virtual Observatory (SVO).
(ii)Set up a prototype for a SVO archive.
sWP50.1 Data Reduction Pipelines
The goal of sWP50.1 was to produce data pipelines for the major European solar instruments. In this context, a data pipeline is the software that calibrates and administrates raw data so that it can be scientifically analysed. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories
sWP50.1.2 Data Compression
The goal of this task was to evaluate 3D compression algorithms for solar spectropolarimetric data.
sWP50.1.3 Image restoration
There are two kinds of image restoration methods in regular use for high-resolution solar imaging, particularly data sets with multiple wavelengths and/or polarization states. They are called Multi-object multi-frame blind deconvolution (MOMFBD) and Speckle interferometry with Speckle deconvolution, e.g. with the Kiepenheuer-institute speckle interferometry package (KISIP). One task of sWP50.1.3 was to improve the accessibility of the codes for both methods. The second important task in this sWP was to compare the two methods.
sWP50.2 Solar Virtual Observatory
The primary premise for this work was to produce the SOLARNET “Document on standards for data archiving and VO”.
WP60: Advanced Instrumentation Development
This work-package aims at the development of new instrumentation for existing solar telescopes, enhancing their capabilities and scientific potential. Some of the novel concepts and designs explored in this WP may be applicable to future large-aperture telescopes, as well.
sWP60.1: Large Diameter Etalon development
The purpose of task sWP60.1 is to build a prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI), with minimal cavity defects. The work done in this sWP represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
sWP60.2 Image Slicer for 2D Spectroscopy
Integral Field Spectroscopy (IFS) is a technique that provides the spectra of all the points in a bidimensional field of view at the same time and under the same conditions using an Integral Field Unit (IFU). Its application offers information along three dimensions: X, Y, λ. There exist different alternatives of Integral Field Units (optical fibers, microlenses and image slicers). In this sWP the latter option is explored with a prototype construction and testing. Task sWP60.3 has developed the microlenses alternative (see below). The objective of sWP60.2 was to develop a prototype of an IFU based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope, to make solar observations obtaining the spectra of a 2-D region simultaneously and to validate the concept for the European Solar Telescope instrumentation.
sWP60.4 Fast Imaging Polarimeter
The goal of the Fast Solar Polarimeter (FSP) project was the development of a novel solar imaging polarimeter with an emphasis on significantly increased polarimetric accuracy and high spatial resolution.
WP70: Wavefront control: turbulence characterization and correction
This work-package aims at measuring the turbulence surrounding telescopes and at determining the measures that can minimise its effects on image quality.
WP70 is divided into 3 sWPs, with a total of 5 tasks, two related with adaptive optics issues (sWP70.1) one related to seeing measurements (sWP70.2) and the other two related to seeing effects minimization (sWP70.3):
sWP70.1. Adaptive Optics (AO)
Task sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
Task sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
sWP70.2. Atmospheric Seeing Characterization
sWP70.3. Local Seeing
Task sWP70.3.1 Application of CFD techniques for local seeing optimization
Task sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
WP80. Synoptic observations: Solar Physics Research Integrated Network Group (SPRING)
Anetwork of telescopes with a small aperture but a large field-of-view can provide useful data to prepare observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures. The goal of this work-package has been to define the scientific requirements of such a network and to analyse the technical alternatives that can deliver the required data products of interest.
WP90. Access Programme to key ground-based infrastructures
The goal of this WP is to provide access to external users to the facilities included in the project (THEMIS, SST, VTT, GREGOR, IBIS@DST and ROSA@DST).
WP100. Access to Science Data Centres. Space missions (led by UiO)
Access to the most demanded European Science Data Centre, providing data gathered by the solar satellite HINODE, and the Solar Dynamics Observatory (SDO), will be offered. As previously mentioned, this access to space-based data for solar research, offered for the first time under an I3 supported by the EC, will expand significantly the content of the Programmes for the high-resolution solar physics community supported in the past.
The Hinode Science Data Centre Europe, the Belgian Web Incessant Screening and the German Science Center for the Solar Dynamics Observatory, are the three facilities offering access and services under this WP101, WP102 and WP103.
The main efforts of this WP, concerning these web-based databases, will be to widely advertise these resources, to increase the number of EU researchers accessing to them.
Project Results:
WP10: Project Coordination and Management
Under this WP, the following activities have been carried out:
-Board and Executive Committees Meetings. All documents of these meetings are available at the private pages of the SOLARNET webpage.
-Organisation of meetings and events: 33 meetings or related events were organised.
-Distribution of the payments to the partners: All EU funds were readily distributed to the partners after reception at the IAC
-Follow-up of deliverables and milestones: A total of 25 milestones were achieved and 72 deliverables produced
-Regular interaction with SOLARNET partners and WP and sub-WP leaders.
-Support to the Transnational Access and Service Programme. 59 astronomers supported with travel and subsistence grants to carry out observations in situ in the Canary Islands telescopes.
-Support to activities related to the Solar Physics Networking: four meetings, five SOLARNET Schools and Workshops, SOLARNET Mobility of Young Researchers Programme.
-Outreach aimed at reaching the goal of promoting joint actions specifically designed to enhance the visibility of the project at international level and getting the targeted audience at large.
-Follow-up of the use of resources. The total human effort has reached approximately 104.58 % of the total amount of manpower dedication expected for the four-year SOLARNET period (689 p-m).
WP20: Integrated operation and exploitation of solar physics facilities and coordination with other research infrastructures
The main body of the work package is the Forum for Access and Services (FAS). At the meetings of the FAS all objectives of this work-package have been monitored and discussed. All telescope and instrument owners have formed part of the FAS, and participated at its meetings. The meetings have been open to all people involved in relevant SOLARNET work- packages.
The EAST TAC has allocated jointly all SOLARNET ACCESS time to the various telescopes and instruments. In order to adopt the EAST TAC to its new challenges, the FAS decided at its first meeting to assign as EAST TAC members one representative of each institutions that operate the telescopes/instruments handled by SOLARNET (GREGOR, VTT, SST, THEMIS, IBIS@DST and ROSA@DST), plus one representative of IAC and NSO. EAST approved this assignment at its General Assembly 2013.
The EAST TAC granted observing time in PI mode as well as in the ‘service’ mode. The PI mode is the traditional observing mode, in which the principal investigator and parts of his/her team goes to the telescope and performs the observations. The technical support is typically limited to the telescope and the instruments need to be operated by the observing team. The service mode is a novel observing mode in solar physics. In this mode the proposers describe the scientific objective and give a detailed explanation on what data they need. The data is acquired by a service team at the telescope following the requirements that the proposers have specified.
The EAST TAC announced the call for proposals for the observing seasons since 2013 to 2016. The calls were issued in ‘SOLAR NEWS’, which is subscribed by most Solar Physicists worldwide. Also, the EAST TAC informed all ‘known’ people that could be interested in the ACCESS program. In addition the Project office implemented a web page within the SOLARNET internet portal to promote the ACCESS program, and distributed a poster on the TAS. The proposal deadlines were in mid-January of each observing season. The number of submitted proposal was higher than expected. The EAST TAC checked the proposals for eligibility and contacted the telescope operators for checking the technical feasibility. The EAST TAC selected two independent referees who rated the proposals according to their scientific merit. Based on the scientific merit, the EAST TAC granted the observing time.
The ACCESS programme was most successful in being oversubscribed and bringing new users to the telescopes. In the course of the 4-year program, the oversubscription rate increased steadily. In 2016 - the fourth year - the ACCESS program was oversubscribed by 200% (26 proposals for 13 allocated projects). If only PI mode campaigns are considered, the oversubscription rate is even higher. It amounts to 250% (22 proposals for 9 allocated projects). During the whole project, the EAST TAC allocated a total of 68 observing campaigns with 466 observing days. 353 users from 18 countries were awarded with observing time and 59 received a travel & subsistence grant to perform the observations. 30 of 59 of them were new users. The figures above show some statistics of the access programme.
Presentations and discussions on the four consecutive FAS meetings brought together all the data pipeline experts from the SOLARNET telescopes, as well as US data scientists that are involved in the data pipelines of the upcoming 4m solar telescope DKIST in Hawaii. Having brought together this community is a major success of WP20. The exchange of expertise in the design of data pipelines between the different groups manifests a most valuable result of the WP20 endeavour. Some aspects of this undertaking are summarized in the 'Final report on data pipeline guidelines'. The concept for a data pipeline needs to be distinguished, depending on the type of instrument: each type of instrument needs its own characteristic design.
A document to define a standard for the metadata in Solar Physics has been produced. The standard has evolved continuously and the document has been updated whenever considered relevant. The latest version can be downloaded at http://sdc.uio.no/open/solarnet-20.3/. In the community it is known as the SOLARNET standard for metadata and is most relevant to the community world-wide. The team of the US 4m solar telescope DKIST who is designing the concept for their data pipelines is also taking advantage of 'our' European SOLARNET standard.
In this sub-work package we have achieved another major breakthrough: the ingestion of ground-based high-resolution spectro-polarimetric data into the SVO prototype. This was made possible, because during the SOLARNET project, data pipelines and data archives adopted the SOLARNET standard and data archives for various SOLARNET telescopes and instruments were developed. For example, the near-IR spectro-polarimeter GRIS@GREGOR now offers a quick look data archive that provides science-ready data for external users: http://archive.leibniz-kis.de/pub/gris/. The GRIS instrument data pipelines provides the necessary metadata that is ingested into the SVO prototype, such that the GRIS data is fully searchable by external scientists. A similar data archive is provided by BBI@GREGOR, GFPI@GREGOR, and the Italian IBIS@DST data. Their metadata is also ingested into the SVO prototype.
In dedicated campaigns, facilities world-wide were coordinated to perform a joint scientific observation campaign. Progress in solar science relies on multi-instrument and multi-wavelength observations assisted by spectro-polarimetric observations with large field-of-views and long temporal coverage. Within this task, a number of such campaigns was performed and evaluated, using ground-based high-resolution telescopes in optical, near-IR, and radio wavelengths. Theses campaigns were supplemented by high-resolution and synoptic space-based missions like SDO, IRIS, STEREO, and Hinode.
New images taken with the Atacama Large Millimeter/submillimeter Array (ALMA) in Chile have revealed otherwise invisible details of our Sun, including a new view of the dark, contorted centre of a sunspot that is nearly twice the diameter of the Earth. The images are the first ever made of the Sun with a facility where ESO is a partner, with a substantial contribution of Dr. Roman Brajsa (HVAR) who acknowledges funding from SOLARNET. The results are an important expansion of the range of observations that can be used to probe the physics of our nearest star. The ALMA antennas had been carefully designed so they could image the Sun without being damaged by the intense heat of the focussed light. The corresponding press release can be found at http://www.eso.org/public/news/eso1703/ .
Prior to SOLARNET solar observing campaigns were executed in PI mode. A Principal Investigator travels to the telescope and performs the observations with a very limited level of technical assistance. This led to a distinct scientific career of an 'observer', since those PI needed to be acquainted with the optics and the instruments of a telescope, and how to use the sub-systems. As these telescope systems became more and more complex the observing task can no longer be performed by a solar physicist whose expertise focuses on solar physics. Therefore, novel observing modes needed to be developed in which a team of instrument experts carry out the observations and calibrate the data while the solar physicist analyses the data to the benefit of science. In the novel queue observing mode, observing proposals are evaluated and ranked by a time allocation committee, and executed by dedicated telescope staff.
During SOLARNET we gathered experience in operating solar telescopes in queued service mode performing dedicated campaigns at DST, SST, and GREGOR. Observing campaigns were queued according to scientific merit and executed based on several parameters like, e.g. target availability, seeing conditions, and instrumental configuration.
Valuable insights in practical realization issues were gained which will direct the implementation of the queued service mode at the planned European Solar Telescope (EST).
WP30: Solar Physics Networking
This work-package is divided in three sub-Workpackages:
-sWP30.1 Meetings
-sWP30.2 Mobility
-sWP30.3 Training (Schools and Thematic Workshops)
sWP30.1: Meetings on solar physics
Four meetings were organised by SOLARNET.
The 1st SOLARNET Meeting “Synergies between ground- and space-based solar research” was held in Oslo on 5 – 8 August 2013 (http://folk.uio.no/matsc/oslo-13/index.html ) together with the 3rd EAST / ATST Meeting. The meeting had 67 participants. The on-line Proceedings of the Meeting are available at http://folk.uio.no/matsc/oslo-13/program.html. Exemption of the registration fee was given to 26 young participants.
The 2nd SOLARNET Meeting "Solar and stellar magnetic activity" was held in Palermo, Italy, on 2-5 Feb. 2015 (http://www.astropa.inaf.it/Solarnet2015/Solarnet2015.html). Specific aims of this meeting were to review the current understanding of magnetic fields in the Sun and similar stars and to discuss future directions of research. The meeting was attended by 61 participants. The proceedings of the meeting are available at http://www.astropa.unipa.it/Solarnet2015/Proceedings/Proceedings.html. Exemption of the registration fee was given to 12 young participants.
The 3rd Solarnet Meeting “Solarnet III / HELAS VII / SpaceInn: The Sun, the stars and solar-stellar relations” was held in Freiburg 31 August – 4 September 2015 (http://www.iac.es/congreso/solarnet-3meeting/) together with the 7th International Meeting of the European Helio- and Asteroseismology Network (HELAS) and its project SpaceInn (Exploitation of Space Data for Innovative Helio- and Asteroseismology). The goals of the conference were to review the state of knowledge in helioseismology, which studies the interior of the Sun through observations of the waves observed at the surface, and the application of its tools and techniques to other stars, in the so called asteroseismology. The meeting had 84 participants. In addition to the plenary sessions, four splinter meetings were held. The meeting was covered in Twitter. All posts related to the conference can be found via #cid_solarnet3_2015. In order to reach out the broad scientific community and the public, all material presented at the conference is made available on the Open Access internet platform www.science-media.org. Financial support, in form of waiver of the registration fee and/or travel support was given to 14 young participants.
The 4th Solarnet Meeting “The physics of the Sun from the interior to the outer atmosphere” was held in Arrecife, Lanzarote (Spain) from 16th to 20th January 2017, during the Third Reporting Period. The purpose of this conference was to provide a coherent picture of the Sun as a single physical system playing all the underlying physical processes measured and observed in the solar atmosphere to date. The meeting had 143 participants. Financial support, in form of waiver of the registration fee and/or travel support, was given to 26 young participants. The meeting was covered in Twitter. The posts related to the conference can be found via #solarnet4m. The presentations and the posters can be downloaded at the following link:
http://www.iac.es/congreso/solarnet-4meeting/pages/talk-and-poster-download.php
sWP30.2: Mobility
SOLARNET supported the mobility of young researchers as part of the networking activities in this work-package. The mobility programme has been designed, as an additional aspect of the training program, to reinforce the contacts between different groups and to allow young researchers to begin early to establish international collaborations.
This task was supervised by the Mobility Evaluation Committee (MEC), a group especially created within SOLARNET, which evaluated and selected the submitted applications. The program supported travel, accommodation and subsistence costs up to a given amount per week. The application was always open and a deadline was set every six months. At that moment, the MEC met and decided, based on a combination of curricula and quality of the project, the young students whose visit could be funded. In order to obtain a better distribution of participants in the framework of the gender aspect, the poster advertising the last announcement included a statement inviting explicitely young female scientists to apply. The Mobility program, characterized by 7 Calls during the period March 2013-March 2016, has been a definite success, with a high number of applicants (45). One of the big successes of the program has been to use the available funds to cover more young researchers than initially expected.
Some statistical figures are:
-20 visits in 7 calls (5 women)
-range of duration: 4 – 14 weeks
-total weeks: 165
-average duration: 8.3 weeks
-average cost per student: 2440 €
-average cost per week: 251 €
-total costs 49826,29 € (planned 35200 €)
-5 – 6 papers in preparation/submitted/accepted
It is also worth to stress that the applicants to the last Call were all young female scientists, ensuring that the gender issue in the end was more balanced than before the last Call. All funded young researchers and most of the host institutions that they have visited have submitted brief reports summarizing the work carried out during the stay. This was one of the deliverables planned for SOLARNET. The high level of the collaborations clearly demonstrates the success of the program.
sWP30.3: Training (Schools and Thematic Workshops)
The 1st SOLARNET School was held in Wroclaw (Poland) from March 24 to April 4, 2014 (http://school.astro.uni.wroc.pl) and was dedicated to PhD students and novel post-doc researchers who wanted to broaden their knowledge on solar physics. In addition to the lectures, several "hands-on" sessions were planned, coupled to the appropriate lectures, in order to allow the participants to develop their practical skills. There were 40 applications for the school participation submitted to the Scientific Organizing Committee and 20 young researchers were finally selected. During the School, some lectures on complementary skills were provided, in order to describe the practical aspects in the work of researcher. The presentations and the materials provided by the lecturers are available at the following webpage: http://school.astro.uni.wroc.pl/materials.php . During the School each student gave a short presentation on her/his scientific work (around 15 minutes); all the presentations are available at http://school.astro.uni.wroc.pl/presentations.php . In parallel with the school, the SOLARNET workshop "Radiative processes in the Sun and stars" was also organized in order to allow the young researchers to participate in splinter sessions and to discuss their research with experienced scientists. The main aim of the workshop was to present and discuss the recent scientific results concerning different aspects of solar and stellar phenomena.
The 2nd SOLARNET School "Ground- and space- based solar instruments" (see http://www.astro.sk/SOLARNET_2ND_SCHOOL/solarnet) was held in Tatranska Lomnica (Slovakia), in October 5-16, 2014. The number of applications was 19 and 12 students were selected to attend the School. The lectures, carried out by experienced scientists, were aimed at giving students knowledge on current and future ground- and space- based solar instruments, observational strategies, post-focus instrumentation and data acquisition. The hands-on sessions were carried out also at Lomnický štít Observatory on the Double Solar Coronagraph and Coronal Multi-channel Polarimeter. There was information presented on the European Solar Telescope (EST) project and on the FP7 SOLARNET project, as well as lectures on complementary skills. In parallel with the school, the Workshop: "Methods in high resolution and synoptic solar physics" and 2nd SPRING (Solar Physics Research Integrated Network Group) were also organized in the same place, Slovakia (see the page http://www.astro.sk/SOLARNET_2ND_WORKSHOP/ for a detailed description of the program).
The 3rd SOLARNET School on “Solar Magnetic Fields: Modeling and Measuring Techniques” was held in Granada (Spain) between the 18th and the 23rd of May, 2015. The goal of this school was to provide PhD students and early career potsdocs with a solid background in solar spectropolarimetry, radiative transfer of polarized light and diagnostic techniques. The programme combined both theory and hands-on activities to achieve that goal. Lecture slides, as well as the necessary inversion codes for the hands-on exercises, were provided for downloading at the School’s website (http://spg.iaa.es/School). A total of 26 young scientists, comprising PhD candidates and early career postdocs from more than 10 countries, were selected to participate in the school (out of 32 applicants). Students were invited to stay one more week and attend the 3rd SOLARNET Workshop, to be held the following week, with the financial support from SOLARNET. The 3rd SOLARNET Workshop was held in Granada (Spain) since 25th to 28th of May, 2015. (http://granada-en.congresoseci.com/polarization2015/index).
The 4th SOLARNET School on “Solar MHD and Reconnection” was held at MSSL (Surrey, UK), 13th-19th of April, 2016 (http://www.ucl.ac.uk/mssl/solar/SOLARNET-4 ). The aim of this school was to provide students and early-career researchers with a solid grounding in Solar MHD and Reconnection using a combination of lectures and hands-on analysis sessions. The School was attended by 41 participants (16 women); further, 39 % of the speakers were women. Lecture slides (https://www.ucl.ac.uk/mssl/solar/SOLARNET-4/school-programme) were provided for downloading from the School’s website and are still available at the same link. The associated thematic Workshop on “Solar eruptive events: observations and modeling” (http://www.ucl.ac.uk/mssl/solar/SOLARNET-4/workshop-programme) was held between 20th and 22nd April. During the workshop the students attending the School were invited to present their research.
The 5th SOLARNET School on “Waves and oscillations in the solar atmosphere” and the thematic Workshop on “Heating Mechanisms in the Solar Atmosphere” were held at Queen’s University Belfast from the 25th of August to the 2nd of September 2016 (https://star.pst.qub.ac.uk/wiki/doku.php/public/solarnet5/start ). The School was attended by 25 early career researchers from 10 different countries (UK, Germany, Austria, Belgium, Norway, Sweden, Spain, USA, Algeria, Georgia. Among these, 11 attendees were women. Lecture slides, as well as the exercises and the necessary inversion codes for the hands-on activities, were provided for downloading from the School’s website (https://star.pst.qub.ac.uk/wiki/doku.php/public/solarnet5/start ) and are still available at the same link. The 5th SOLARNET Workshop was held in Belfast (UK) between the 31st August and the 2nd of September, 2016. The Workshop was attended by 48 researchers from 13 different countries (UK, Germany, Austria, Belgium, Italy, Norway, Sweden, Spain, USA, Algeria, Georgia, India, Bulgaria). Among these, 23 attendees were early career researchers and 15 attendees were women.
WP40: Innovation Towards Industry
To reach the objectives of this work-package, the following tasks were addressed:
- Identification and analysis of technologies, techniques and new concepts, through the development of a database for technology offers, consortium knowledge base and the technology needs identified by the solar astrophysics community
- Dissemination of the innovative aspects and findings of the project towards industry at international/European events, SOLARNET workshops, European workshops, ...
-Organisation of workshops related to technology transfer: (i) 1st Solarnet Technology Transfer Workshop linked with SOFT 2014 Fusion Conference (San Sebastian, Spain). It showed several technologies/needs with potential solutions/ collaborations within the fusion industry; (ii) 2nd Solarnet Technology Transfer Workshop was held during this period. Linked with LSW London Space Week, B2B and Summit Conference (London, UK). It showed a significant number of technologies/needs with potential solutions/collaborations within the space industry; (iii) o3rd Solarnet Technology Transfer Workshop between SOLARNET and ESA-ESOC (July 2016). Presentation of SOLARNET project and relation to EST. Presentation of EST/SOLARNET Technology Offers/Needs to ESOC. ESOC presented requirements related to future Optical Ground Stations.
- Assessment of space technologies: (i) Tecnalia has access to the ESA European Technology Transfer Network database; (ii) continuous monitoring of space tech-nologies for potential application in solar astrophysics community; (iii) mMonitoring of opportunities for transfer out from ground based solar telescopes to space based or other sector applications; (iv) Monitoring of Invitation To Tender (ITT) opportunities through the ESA EMITS portal.
- Dissemination of the project and specifically the technology offer and needs listings: (i) Numerous Spanish space industry meetings; (ii) Participation at International/European Events; (iii) Three SOLARNET Industrial workshops; (iv) Continued interaction with European networks (including ESA); (v) Fact finding mission to EC (Brussels); (vi) Continued development of national industrial contacts.
- Identification of future projects: Related to 3rd Industrial Workshop, WP40 has identified future potential ESA GSTP projects related to the development of the optical ground station that should be further assessed by members of the SOLARNET community for future funding opportunities: (i) Laser ranger (approx. budget 7-8 M€); (ii) Low Cost Meter-Class Adaptive Optics Communications Breadboard (1M€); (iii) Deep Space Low Cost 4m monolithic Optical Antenna for Day/Night Operations (400k€); (iv) 4m mirror concept (approx. budget 20M€).
- Promotion of synergies with other I3 initiatives on this same topic. The main contact with I3 initiatives has been with ASTERICS (www.asterics2020.eu). The aim of the collaboration was the management and exchange of large volumes of data related to ESFRI projects.
WP50: Tools for innovative data handling: pipelines, databases and SVO
To reach its objectives, the WP50 was divided into 4 tasks grouped within two areas corresponding to the main objectives:
sWP50.1: Data Reduction Pipelines
Task WP50.1.1: General pipeline work
Task WP50.1.2: Data compression
Task WP50.1.3: Image restoration
sWP50.2: Solar Virtual Observatory
sWP50.1 Data Reduction Pipelines
Task sWP50.1.1 General pipeline work
The goal of sWP50.1 was to produce data pipelines for the major European solar instruments. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories.
As input for the pipeline work, SOLARNET sWP20.2 issued guidelines with general recommendations. The most important of these was that output data should be in the FITS format with metadata included as standardised headers. Metadata are data that describe the circumstances of the observations and how the data have been processed. It was the task of SOLARNET sWP20.3 to produce recommendations for the FITS headers and they were issued in the corresponding report and much of the work addressed here was devoted to adapting pipelines to these recommendations. This is no small task. Being fully compliant with the recommendations means introducing 99 new keywords and their values while making sure that no other keyword is used in a conflicting way. At the same time auxiliary data such as wavelength scales would have to be treated and packaged according to new specifications. The SOLARNET recommendations include a self-declaration of its level of compliancy. Thus SOLARNET=1.0 means fully compliant and SOLARNET=0.5 means partially compliant (some fundamental keywords exist and there are no conflicts between other keywords and the SOLARNET recommendations). Perhaps the most important legacy of this effort is that the community has learnt how important these issues are and how they should be tackled.
The following is a summary of work being done for all instruments under the umbrella of SOLARNET.
GRIS (IAC): This pipeline is almost completely automatic, only demanding a minimum of input from the user. The output data are now very close to being SOLARNET compliant. As GRIS produces spectrograms, there is no image reconstruction technique involved.
BLISS/HiFI (AIP): When SOLARNET started, BLISS was a planned instrument for the GREGOR telescope. It was later cancelled but replaced by HiFI which is a context imager for blue light that is less ambitious than the projected BLISS. The SOLARNET Executive Committee approved that the software development for HiFI replaces that for BLISS. HiFI is one of the instruments that use the data pipeline sTools. HiFI metadata is conforming to the SOLARNET=0.5 level. A technical obstacle for reaching detailed compliancy is that the GREGOR telescope pointing recording has uncertainties, precluding exact automatic recording of the position on the solar disc. Image reconstruction using Speckle.
GFPI (AIP): The pipeline software is sTools which is also used for HiFI. Once the HiFI metadata treatment has been verified to comply with recommendations the procedure will be transferred to GFPI data. The work with GFPI data is more complicated than for HiFI. Image reconstruction using MOMFBD.
CRISP, CHROMIS (SU): CRISPRED, the pipeline for CRISP data, was developed during the first half of the SOLARNET. It has played an important role in solar physics for allowing several researchers to use CRISP. CRISPRED has active users in twelve institutes (as judged from the subscriptions for the mailing list for CRISPRED/CHROMISRED users). CHROMIS was installed at the SST in August 2016 and the first observations were performed during the following two months. This meant that much of the effort during the last months of the SOLARNET project was spent on producing its pipeline: CHROMISRED. This new pipeline was built with similar and partly overlapping software as the existing CRISPRED for CRISP. It has more than 50000 lines of code, not counting the image-reconstruction part. CHROMISRED is close to becoming fully compliant with the SOLARNET metadata standards (SOLARNET=1.0). After it has been found stable, CRISPRED will be taken to that level too as the two pipelines are merged. Both pipelines involve MOMFBD image reconstruction.
TRIPPEL (SU): This instrument is arguably the least standardised in the whole suite since observing campaigns make different choices for detectors and slit-jaw imaging setups. Thus many changes are made between different data sets, so much that it is not enough to set a few input parameters. The goal has been that such changes should be confined to a minimum number subroutines.
LARS (KIS): The instrument was upgraded in 2016 from a prototype to regular-use instrument. Starting with the 2017 observing season, LARS can be used also by non-expert observers. Metadata treatment is extensive and very close to being SOLARNET compliant though the formal declaration has not been made. Technical limitations of the telescope preclude automatic and precise recording of pointing coordinates on the solar disc.
TESOS (KIS): The metadata treatment was reworked and reached SOLARNET=0.5. Image reconstruction using Speckle.
ROSA (QUB): The work in the ROSA pipeline include introducing metadata treatment to the SOLARNET=0.5 level and writing a manual. The pipeline relies on speckle image reconstruction using KISIP.
IBIS (INAF): The IBIS pipeline exists in an INAF and an NSO branch. The INAF branch has undergone a metadata-treatment push to reach SOLARNET=0.5. In parallel, an archive for IBIS data has been built: http://ibis.oa-roma.inaf.it/IBISA/ . Apart from making IBIS data available to the community this way, this effort has also driven and motivated the pipeline work. Image reconstruction using MOMFBD.
TUNIS and MTR (CNRS): Level 0 metadata are almost completely compliant with the recommendations but have yet to be propagated to the fully reduced data.
It can be concluded that, thanks to the work in task WP50.1.1 there are now modern pipelines for all major European solar instruments. The pipelines have been raised to a new standard where they are more reliable and easy to use. This has led to higher productivity and more and better science. The work with conforming the pipelines to the needs of virtual observatories will allow the future development of such databases.
Task sWP50.1.2 Data Compression
Comparative tests carried out during the EST Design Study indicated that application of state-of-the-art 2D compression methods to solar spectro-polarimetric data leads to a final data volume that range between 1/2 (lossless compression) to 1/5-1/10 (lossy compression) of the original data volume. However, high-resolution spectroscopic and spectro-polarimetric solar observations consist of time series with small changes from one frame to the next. The correlation present in the spectro-polarimetric data-sets over all dimensions, spatial, temporal and spectral, suggests that the use of 3D compression algorithms applied to either (x,y,λ) or (x,y,t) data cubes may allow an enhancement of compression efficiency with respect to 2D methods, with no information loss.
Solar imaging data have, in general, peculiar characteristics, such as high S/N, continuous data on the FoV, etc., which are not always typical of other hyperspectral data-sets. The data-sets used in the Report on lossless JP3D compression of solar data-cubes are a partial description of the present variety of high resolution photospheric solar data, which will probably be maintained in the next years. The two data-sets are retrievable from https://www.fisica.uniroma2.it/~solare/en/?p=257 to allow anyone to compare the compression performances of different algorithms on the same data-sets.
To extend the analysis on chromospheric solar data, in Report on lossless JP3D compression of solar chromospheric data-cubes, we compressed and de-compressed a spectroscopic scan of a chromospheric line consisting in 80 repetition of a 21-point imaging scan of the CaII 854.2 nm line of the same Active Region. These data-sets include calibrated broad-band, narrow-band and spectropolarimetric observations of the photosphere and chromosphere. They also represent different types of targets: a Quiet Sun region at disk centre and an Active Region at disk centre. Therefore, they allow us to evaluate the performance of the compressing algorithm in different, but ’typical’ situations of solar observations: low-contrast images with localized small-scale magnetic fields, high contrast images with concentrated and diffuse magnetic field, highly dynamic chromospheric datasets, etc.
The compression methods studied constitute a selection among the most frequently adopted image compression procedures in a variety of fields of application; they exploit in different ways the properties of the Discrete Wavelet Transforms often coupled with standard entropy coders or similar coding procedures applied to the different bit planes in order to allow a progressive handling of the original image.
2-D lossless compression algorithms were able to reach a compression factor between 1.5 and 3 for the various types of solar images used. Among these algorithms, the one with the best compression ratios was the SPIHT, the one with the shortest compression+decompression times was the JPEG2000. Among the lossy 2D compression algorithms, the JPEG2000 had the best performance overall. The 3D compression algorithms allow us to gain a 2x factor over the 2D methods in the compression ratio with apparently no data degradation. Of course, this comes with a cost, namely the 3D compression is more demanding in hardware and more time consuming. Nevertheless, the evolution of the computing capacity is fast enough to allow us to perform the compression of large data-cubes with relative ease, and, above all, the computing capabilities are evolving faster than the storage or the data-transfer capabilities. Since the differences in the performances of these 3-D methods were within the errors, we choose JP3D because it is open-source and well documented. We applied loss-less JP3D compression to calibrated solar data-cubes of different types. The data-sets employed are a partial description of the present variety of solar data, which will probably be maintained in the next years. The performance of the 3D compression varies with the data-type: G-band are the most compressible (~3 BPV), while Stokes I are the less compressible (~7 BPV). The gain against a 2D compression varies from -50% to -30% of the original data volume. The gain in ordering data in [x,y,λ,t] or [x,y,t,λ] is apparently negligible. The JP3D algorithm is efficient in handling the larger files, with little differences in compressing a single large data-cube or several smaller data-cube.
As conclusion, the JP3D compression algorithm applied to either the (x,y,λ) or (x,y,t) allows an approximate doubling of compression efficiency on solar spectro-polarimetric data by 3-D algorithms with respect to 2D methods on same data. The performance of the compression varies with the data-type: G-band images are the most compressible (~3 BPV), while Stokes I are the less compressible (~7 BPV). We note that this performance is probably not enough to keep data volumes at manageable size (the target compression ratio should be of the order of 25x, i.e. ~1 BPV) therefore, we suggest to explore also the JP3D algorithm performances in lossy compression, for those cases where the preservation of all the information is not critical. We note also that, even considering an enhancement of the data processing capabilities in the foreseeable future, the compression time will be a crucial factor. In fact, the acquisition rates are likely to increase up to 100 1024×1024 pixel images per second and real time compression of the data-sets may not be feasible. This has to be taken in consideration in the observatory daily schedule.
Task sWP50.1.3 Image restoration
There are two kinds of image restoration methods in regular use for high-resolution solar imaging, particularly data sets with multiple wavelengths and/or polarization states. They are called Multi-object multi-frame blind deconvolution (MOMFBD) and Speckle interferometry with Speckle deconvolution, e.g. with the Kiepenheuer-institute speckle interferometry package (KISIP).
One task of sWP50.1.3 was to improve the accessibility of the codes for both methods. SU has made a fork of the MOMFBD code, and rewritten it extensively to make it easier to maintain (using more standard libraries in place of original code), adapted to the data format of the recently installed SST/CHROMIS instrument, and to implement a number of features that should improve the restorations. The new version, redux, is under version control with git and available for anybody at git://dubshen.astro.su.se/hillberg/redux. The speckle interferometry code KISIP is likewise under git version control and available from https://bitbucket.org/fwoeger/kisip.git. Both momfbd/redux and KISIP are integrated parts of data processing pipelines developed under SOLARNET, where standard parameter settings are provided by the pipeline maintainers.
Another important task in this sWP was to compare the two methods. While the comparison is interesting in itself, it also spurs everyone involved to understand the codes better. The method-comparison experiment at the 1st CASSDA-SOLARNET Workshop taught us that in order to interpret the results of such a comparison, it is important to have better control over the assumptions imposed by the calibrations and other processing done in different ways in the different data pipelines. Further efforts were needed to isolate image restoration effects from other steps in the pipelines.
A more carefully designed experiment was started. We installed the KISIP code as a drop-in replacement for MOMFBD in the CRISPRED pipeline, the data processing pipeline for the CRISP instrument at SST. This ensured that all processing before and after the actual image restoration is done in the same way (and in the best way known for that instrument), thus making the comparison more relevant. Using data from the SST, which is a refractor, removes uncertainties in, e.g. how the central obscuration of reflectors like GREGOR and the elongated pupil of VTT affect the image restoration.
Suitable data for this experiment were collected but we soon realized that the theory for one of the calibrations required for SI with AO compensated data had only been developed for AO systems where the deformable mirror is controlled with Zernike polynomials as modes. The SST AO uses control modes that are designed to take both the statistics of atmospheric turbulence and the mirror electrode pattern into account. We have worked to fill in this gap in the theory and a new paper on the subject will be submitted to Journal of the Optical Society of America A shortly.
sWP50.2 Solar Virtual Observatory
Work was concentrated on the data archive prototype linked to the SVO and the implementation of the recommendations in the pipeline work. The primary premise for this work is the SOLARNET “Document on standards for data archiving and VO”. Substantial work was carried out testing the ideas included in this document and helping pipeline teams with the implementation of the recommendations and modifying the recommendations based on feedback from the pipeline implementations.
Given this background, the design principles for the SVO archive were worked out, and it was determined that a highly normalised data-base structure should be used, in order to minimise the maintenance work and the potential for inconsistencies in the data set. Also, it was decided to implement one database per instrument/data set, and a subset of available metadata for each data set was selected for inclusion in the data base. On top of this structure, it was determined to have a database containing descriptions of each subset. Another design choice was to use a RESTful API to access the data base, upon which it would be easy for other applications to be built (also by third party providers).
The data base implementation was made in PostgreSQL. On top of that (and the RESTful access layer), three clients to access the contents of the data base were made: One web application, one IDL API, and one python API.
Subsets of data sets (AIA, EIT, HMI, SWAP, and Themis) were earlier in the database. In this period, further ground-based datasets have been added to the prototype database (Chrotel, GRIS, IBIS, ROSA). The prototype SVO is maintained by ROB and can be studied at http://solarnet.oma.be/.
Another piece of work that was not immediately included in the prototype SVO but will be useful for all coming SVO is the addressing of the problem of listing events. A VO user is likely to search for a specific type of event or features on the Sun, but defining and cataloguing events is not a trivial matter. Based on capabilities developed under another FP7 project, HELIO (Heliophysics Integrated Observatory; Grant No. 238969), we have tried to enhance the criteria used to select data that are based of the occurrence of an event or phenomena, or the presence of a feature. HELIO is based on a service-oriented architecture where the required capabilities are implemented as independent services; there are many instances of most services with a Registry describing their access. This style of architecture means that the SOLARNET SVO can pick and choose which capabilities it wishes to use.
On the data side, we have enhanced the data search capabilities of HELIO to provide access to data sets that complement those that can be addressed through the SOLARNET SVO. HELIO was able to access a wide range of instruments though a number of data sources. In expanding the capability we have concentrated on trying to provide near continuous coverage in observations from ground-based observatories in wavelengths that are not (currently) observed from space.
We have provided integrated access to the synoptic networks established as part of the GONG programme: magnetograms and white light images are gathered as part of the helioseismology observations and since 2010 observations have also been made in H-alpha. Since there are six GONG sites spaced around the globe, under good conditions these ought to provide continuous coverage, however, bad weather and scheduled down time can result in gaps. To supplement the coverage in H-alpha provided by GONG, we have added other observatories where the observations are extensive and accessible. Such sites include Kanzelhöhe (Austria), Catania (Italy), HASTA (Argentina) and Big Bear (California, US) and we continue to look for other sites. We have also been trying to improve access to image data made at radio wavelengths. Some progress has been made but coverage is still patchy because.
WP60: Advanced Instrumentation Development
This work-package aims at the development of new instrumentation for existing solar telescopes, enhancing their capabilities and scientific potential. Some of the novel concepts and designs explored in this WP may be applicable to future large-aperture telescopes, as well.
sWP60.1: Large Diameter Etalon development
The purpose of task sWP60.1 is to build a prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI), with minimal cavity defects. A preliminary analysis of the scientific requirements for an FPI-based, imaging spectro-polarimeter, to be used with a 4-meter class solar telescope, identified cavity defects of λ/150 @ 633 nm as a necessary property. The work done in this sWP represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
To date, most of the capacity-controlled, tunable FPIs used in solar physics are produced by IC Optical Systems Ltd (ICOS), with diameters between 50 and 150 mm. After several discussions with the company regarding the possibility to improve their default model and the plausibility of reaching the necessary plate flatness over a large diameter, ICOS was selected for the construction of the SOLARNET prototype. Part of the decision rested also on the fact that the INAF partner of sWP60.1 already owned a CS100 controller (an item of high cost), needed to operate ICOS FPIs.
Besides the intrinsic quality of the optical surfaces, common causes of significant cavity defects for the ICOS etalons include gravity, surface coating, preload stresses, and effects of the piezoelectric actuators used in the capacity-controlled system that regulates the spacing between the two etalon’s plates. The final goal of sWP60.1 was to correctly identify and characterize these defects, as well as to propose a path forward to their minimization. After several iterations with ICOS, a modification of the design was agreed upon in the early stages of the SOLARNET project, leading to a fully symmetric design, with two identical plates. A Finite Element Analysis (FEA) of such design was conducted early in the SOLARNET project, and indicated the possibility to further reduce the cavity defects by using the etalon in a horizontal, rather than vertical, configuration.
The prototype was commissioned to ICOS in spring 2015, with a new and improved design that aims at minimizing pre-operation stresses, due to gravity, pre-load etc. The fabrication of such a large, and delicate piece of equipment is extremely complex, and several issues introduced delays in the expected delivery time. These issues included multiple attempts at optically contacting the wedges, and repeated iterations of the plate polishing, to achieve the required figure. By early September 2016, a conversation with ICOS outlined that the plate figure was still not fully to specification. However, given the imminent closing of the whole SOLARNET project, the delivery of the New Prototype ET150 was concerted “as is”, in order to start testing its actual performances, including stability, repeatability, effects of gravity, etc., with the understanding that the cavity would be less than optimal. A procedure for measuring the cavity defects of the etalon had been already developed at the Optical Measurements and Testing Laboratory CNR – INO, and tested on an older ICOS 50 mm etalon (ET50), property of INAF.
An extensive set of tests and measures of the performance of the ICOS ET150 has been carried out. The tests have been performed at the CNR-INO (Istituto Nazionale di Ottica) Lab in Florence (Italy) using a 6” Zygo GPI XP interferometer and a measuring procedure improved at the same lab. The measurement procedure has characterized the ET150, both as a static and as a dynamic system. The tests were centered on the cavity properties: (i) Cavity shape; (ii) Effects of gravity (horizontal vs vertical configuration); (iii) Cavity shape throughout a full range scan
The setup used to measure these properties and the corresponding results are detailed in the corresponding report, revealing strengths and weaknesses of the prototype. The tests show that changing between horizontal and vertical orientation does not distort the cavity significantly, suggesting that the design is robust. The positioning of the piezoelectric actuators does not seem to have an important effect on the cavity shape, either. Future work should focus on improving the surface flatness and the polishing procedure.
The results point towards the feasibility of a 150 mm, capacity controlled FPI built using “conventional” methods, for use in solar imaging spectroscopy. Remaining sources of concerns are whether the desired flatness (λ/150 at 632.8 nm) can be reached with a more careful polishing procedure, as well as the possibility to effectively reduce or eliminate the residual tilts introduced by the closed capacity-controlled loop. At the end of March 2017, the new ET150 has been returned back to ICOS for final polishing of the etalon plates. Further tests will be performed once the prototype is returned from the manufacturing company.
sWP60.2 Image Slicer for 2D Spectroscopy
Integral Field Spectroscopy (IFS) is a technique that provides the spectra of all the points in a bidimensional field of view at the same time and under the same conditions using an Integral Field Unit (IFU). Its application offers information along three dimensions: X, Y, λ. There exist different alternatives of Integral Field Units (optical fibers, microlenses and image slicers). In this sWP the latter option is explored with a prototype construction and testing. Task sWP60.3 has developed the microlenses alternative (see below).
The objective of sWP60.2 was to develop a prototype of an IFU based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope, to make solar observations obtaining the spectra of a 2-D region simultaneously and to validate the concept for the European Solar Telescope instrumentation.
An image slicer is an optical system placed at a focal plane to divide the image into slices which are later distributed generating one or more long-slits. This generated slit feeds a spectrograph as in the case of conventional spectrographs but, since this slit is a reorganization of a bidimensional region, the spectra of all the points of that 2-D field of view are obtained simultaneously.
Image slicers use arrays of mirrors (for pure reflective image slicers) to reflect each part of the image in a different direction using different orientations (tilt X, tilt Y). These orientations are carefully controlled to distribute the images of each ‘sliced’ part of the field of view into the output slit or slits. Thus, the optical path of each ‘sliced’ part of the field of view is defined by the reflection using one mirror of each array and generating a piece of the generated output slit, which is known as ‘mini-slit’. The design of the image slicer for GREGOR was presented early in the project schedule and later updated, after including modifications required by manufacturing tolerances. The prototype includes not only the image slicer but also the field-of-view scanning system (FoV-SS), optical interface to couple it to GRIS, and the optical bench that holds the system. The image slicer was constructed and finally received in October 2016.
The last period of the project was dedicated to finalize the construction of all opto-mechanical pieces and to test the full unit at the lab and telescope, once the IFU was received. The full system was installed at the telescope in November 2016. The setup used at the lab, the tests at the telescope and the results are described in a SOLARNET document. The tests were devoted to determine the mechanical and optical performance of the whole system: Optical and mechanical alignment of FoV-SS and slicer, scanning system accuracy and repeatability, image quality of the slicer, cross-talk, and stray-light. After all these tests, the successful performance of the whole system was demonstrated. New tests at GREGOR are scheduled for the end of June 2017, together with scientific observations.
sWP60.3 Microlens-fed Spectrograph
Task sWP60.3 addresses the design and construction of a microlens-fed spectrograph to measure simultaneously all points of a given area on the solar surface a given area, as an alternative to the Integral Field Unit based on image slicers described above. A prototype, called MiHI (Microlensed Hyperspectral Imager), has been designed and constructed.
The MiHI is a prototype of a microlens-fed hyperspectral imager, specifically designed for solar observations, and intended to overcome the limitations of traditional solar instrumentation. It simultaneously resolves the spatial and spectral dimensions, thus promising data with a S/N ratio close to the theoretical maximum. The challenges of developing such an instrument for solar applications differ from those for night-time applications, both in terms of the necessary control of the amount of stray-light produced by the spatial extent of the object, as well as the spectral resolution, that must be high enough to resolve individual atomic lines in the solar spectrum with sufficient spectral and spatial resolution to derive the physical conditions in the solar atmosphere in 3 spatial dimensions, as well as in time.
The MiHI solves these challenges by using a specially designed and manufactured microlens array that allows all resolution targets to be achieved simultaneously, with a minimum of experimental elements, while retaining an acceptable instrument transmission. The design considerations can be found in the corresponding SOLARNET document.
The principle of operation of a hyperspectral imager is typically based around a device that reformats the image plane, thus making space for the spectral dimension. The MiHI uses a double sided microlens array to reduce the size of each of its image elements and accomplish this goal. To avoid overlap of the spectra, an ultra-narrow bandpass filter is required. The filter must be as high-order as possible, to minimize the contamination from one image element to the next, and to optimize the signal level over the maximum possible spectral range. Steepness of the flanks of the filter transmission profile, as well as maximum peak transmission is therefore of the utmost importance.
To avoid designing and building a spectrograph in addition to designing and building the image formatter, the MiHI prototype was designed as a drop-in replacement for the slit-box of the existing TRIPPEL spectrograph, currently installed at the Swedish Solar Telescope (SST). A number of tests haven carried during an observing campaign carried out in October 2016 at the Swedish Solar Telescope (SST) with the setup described above. The prototype was installed in front of the TRIPPEL spectrograph, aligned and adjusted, after which some of the basic properties were characterized. All opto-mechanical elements required for the installation of MiHI at the STT were constructed during the third reporting period. Their description and alignment procedures can be found in the corresponding report.
The instrument was found to generously exceed the spectral resolution requirements, and meet the spatial resolution and field-of-view (FOV) requirements, but is somewhat less transparent than intended (25%, where 50% was the target), and shows some signs of minor thermal drifts. The reduction and calibration of the data was found to be challenging, and will require considerable further modelling and observing experience to produce high quality data. All details can be found in the SOLARNET report.
sWP60.4 Fast Imaging Polarimeter
The goal of the Fast Solar Polarimeter (FSP) project was the development of a novel solar imaging polarimeter with an emphasis on significantly increased polarimetric accuracy and high spatial resolution. The instrument is based on a fast polarization modulator synchronized to a high frame-rate and low-noise detector, working at a polarization modulation frequency in the 100 Hz range. This increased modulation frequency suppresses spurious polarization signals induced by external disturbances such as atmospheric turbulence or telescope jitter and avoids crosstalk between polarization states, in particular between linear and circular polarization, which limits the accuracy of today’s slower dual-beam polarimeters. In addition, the fast modulation allows to obtain time series of numerically restored Stokes images at a cadence below the typical solar evolution time scale of magnetic structures at the smallest resolved scales (of the order of 100 km on the Sun, corresponding to evolution time scales in the range 1-10s). The higher polarimetric accuracy that can be achieved with FSP will open up new windows for studying solar magnetic fields and is, in particular, expected to advance studies of the enigmatic magnetic field in the quiet Sun and of chromospheric magnetic fields. The instrument is particularly developed in preparation for the next generation of large aperture solar telescopes like the planned European Solar Telescope (EST).
The FSP modulator and its performance in single-beam configuration have been tested and described in the corresponding report. Another document has been produced in which the polarizing beam splitter (PBS) for the dual-beam configuration is described, together with the test results demonstrating its performance. This high-precision optical device has been invented and designed by MPS, in collaboration with the Canadian optics manufacturer LightMachinery Inc. The PBS was delivered to MPS in December 2016.
The optical and polarimetric performance of the PBS has been tested and compared to the specifications. The setups used to measure these properties and the corresponding results are detailed in the corresponding SOLARNET deliverable. The first scientific application will be at an observing campaign in August 2017 at the IBIS filtergraph instrument of the Dunn Solar Telescope (DST) in New Mexico, USA.
WP70: Wavefront control: turbulence characterization and correction (led by IAC)
WP70 is divided into 3 sWPs, with a total of 5 tasks, two related with adaptive optics issues (sWP70.1) one related to seeing measurements (sWP70.2) and the other two related to seeing effects minimization (sWP70.3):
sWP70.1. Adaptive Optics (AO)
Task sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
Task sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
sWP70.2. Atmospheric Seeing Characterization
sWP70.3. Local Seeing
Task sWP70.3.1 Application of CFD techniques for local seeing optimization
Task sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
sWP70.1.1 Multiconjugate Adaptive Optics (MCAO) Simulations and Tests
A particularity of solar AO is that it uses the solar granulation as a reference; therefore the wavefront sensing is performed using correlations on images with a field of view of about 10”. A sensor collecting such a wide field of view averages wavefront information from different sky directions, affecting the sensing of high altitude turbulence, the sampling of which does not depend anymore on the size of the subapertures only, but rather on the size of the projection of the extended field of view. Understanding this effect is crucial for the design of future solar facilities, i.e. to choose the adequate height of the DMs on MCAO systems, and also to predict the quality of the reconstruction that such system would be able to achieve. For that reason, we have studied wide field sensing and explained the analytical equations that describe the process. The equations have been validated with the results of the numerical simulations run with the code FrIM. A complete error budget has been evaluated, including the generalized fitting error.
The influence of the correction order of turbulent layers in an MCAO system in the visible has been simulated using the physical propagation approach. Static phase screens have been generated following a Kolmogorov statistics. The outcome of this study is that amplitude propagation errors can be neglected, while phase propagation errors due to non-linear effects have to be considered. A perfect cancellation of phase errors is achieved if the MCAO correction is done in the inverse order of turbulence occurrence. Nevertheless, the degradation of the performance is not critical if high turbulent layers are corrected in the order of occurrence.
Specific simulations have been performed under different turbulent conditions to determine the optimum height for the MCAO DMs. Using real atmospheric profiles from ORM and OT, we show that, after correction, a homogeneous Strehl higher than 40% can be achieved over a 1 arcmin FoV for all elevations. These numerical results are validated with the analytical equations and the complete error budget commented above.
For the determination of the optimum conjugated heights of the MCAO DMs, a method to characterize the day time turbulence up to several tens of km is derived. The SHABAR instrument used to determine the diurnal seeing conditions (see sWP70.2) is only sensitive up to a distance of 2-3 km to the turbulent layer. The contribution to the seeing of higher layers is crucial to understand the image degradation and, consequently, other means are required to explore the behavior of these high-altitude layers. To complement the results obtained with the SHABAR, we make use of the instrument SCIDAR, installed at the Observatorio del Teide (OT) and which can derive turbulence profiles during the night. We have also used data taken by regular radiosondes that are launched in Tenerife (twice per day). The three data sets (SHABAR, SCIDAR and radiosonde) have been adequately compared and merged to produce a single turbulence profile which may be considered as typical of the atmospheric conditions at the OT. This profile has been used for the optimization of the MCAO system and presented in a report.
Efforts were also dedicated to forecast the MCAO correction. To that aim, a hardware demonstrator was constructed to test our forecasting algorithm FORS (closed loop FORcasting System). We studied the outcomes of introducing both a simple periodic defocus aberration and a real open loop defocus time sequence acquired at the VTT solar telescope. In both cases, FORS grants a significant performance increase, improving the stability of the system in closed loop conditions, and decreasing the amplitude of the residual uncorrected wavefront aberrations.
sWP70.1.2 Implementation of an AO prototype for THEMIS telescope
AO systems are installed at most existing solar telescopes to improve their image quality, with excellent results in imaging observing modes, but the combination of polarimetric measurements with adaptive optics is extremely challenging and it is not usually fully addressed. Being THEMIS a solar telescope which provides high polarimetric performance, the implementation of an AO prototype to improve its image quality retaining its unique spectropolarimetric capabilities, would open the possibility to specialize this telescope in high resolution in polarimetry. The possibility of performing very high quality polarimetric measurements using AO is one of the challenges of the future large aperture European Solar Telescope (EST) and the implementation of an AO system at THEMIS will provide an excellent bench to test these observing techniques for EST.
A full study has been conducted to implement an adaptive optics system for the THEMIS telescope, taking into account the current technical state of the telescope and the scientific goals derived from the polarimetric usage of the long slit spectrograph attached to the telescope. It shall be clear from the beginning that THEMIS has been specifically designed to be a "calibration-free" polarimetric telescope and that the AO design has been done keeping this important specification valid. The adaptive optics (AO) prototype for THEMIS is composed of two separate and complementary branches. On the one hand, there is the design and construction of the AO optical bench itself, including the wavefront sensor, the pupil deformable mirror and all the accompanying mechanical and optical parts. On the other hand, some changes in the mechanics and optics of the telescope are required to accommodate the adaptive optics system.
During the initial phases of the project, the design of the AO prototype was accomplished and the corresponding reports and deliverables were produced. Later, the detailed design of all changes required at telescope level was produced and providers of all new optical and mechanical elements were contacted and purchase orders submitted. Finally, all these elements have been received and tested at the laboratory during the third reporting period. The design is such that the same optical bench can be moved from the lab to the telescope as it is. New mirrors M2 to M5 of the telescope have been acquired, tested and installed at the telescope with their corresponding mounts. The performance of these new optomechanical parts has proved to be within specifications. Global on-sky testing of the whole system is planned for fall 2017. The adequate reports have been generated where all these achievements are presented.
sWP70.2 Atmospheric Seeing Characterization
Two long-baseline SHadow BAnd Rangers (SHABAR) instruments were deployed by the IAC in the Canary Islands during 2010, one at Observatorio del Teide (OT), in Tenerife island, and another at Observatorio del Roque de los Muchachos (ORM), in La Palma island. The instruments acquire sunlight signal data that can be reduced to produce Cn2 and r0 profiles for the lower atmosphere layers up to some 2-3 Km height. The mission of these instruments, together with other daytime turbulence measurement instruments, is the characterization of the daytime sky in both sites, OT and ORM, in order to select the best location for the European Solar Telescope.
Reports have been presented in which the data acquisition and analysis are described, as well as the main results of the data analysis performed showing the statistics and seeing properties at different heights from the ground and the daily variation for the different seasons of the year. These results are presented for both Canarian observatories: OT and ORM. In the figure below, the variation of the median r0 with height at both observatories is presented. More details can be found in the corresponding document, including the diurnal and seasonal variation.
sWP70.3.1 Application of CFD techniques to the local seeing optimization
We have developied numerical methods to estimate the atmospheric seeing and simulate some test cases of certain configurations of the EST. The main feature of the proposed model is that it is based on the numerical dissipations which arise from a particular version of the Variational Multiscale Method, the Orthogonal Subgrid Scale method. It is a finite element method where one assumes that the solution can be split into large and small scales. The former are considered those captured by the finite element mesh, whereas the latter are considered L2-orthogonal to the former and modeled approximately. The advantage of using this kind of models lies on the fact that, by decomposing the fields of interest into coarse and fine scales, they are able to deal simultaneously with the sources of numerical instabilities and the modeling of turbulent effects.
We have tested the performance of the method in three practical cases, namely a convective boundary layer, the flow inside a transfer optics room, and the flow around a telescope enclosure. In all three cases we have compared our model with the results obtained by using a Smagorinsky and WALE models for evaluating the viscous and thermal dissipations, and, in the convective boundary layer case, with the results presented in the literature. The numerical examples show that the method is capable of doing an accurate estimation of the Cn2 coefficients. This fact does not only provide us with a new numerical tool for the evaluation of the atmospheric seeing but it also adds arguments in favor of the viability of implicit LES methods which rely on the numerical stabilization mechanisms for the modeling of turbulence.
In a second stage, efforts were concentrated on completing the set of CFD simulations to evaluate the seeing conditions in several configurations of the telescope facilities for EST and help the SOLARNET consortium to decide the optimum design of the EST facilities. To that aim, different configurations were analysed for the telescope facilities with a conventional dome (closed configuration) and facilities with windshield (open configuration), in summer, for North wind of 5 m/s, in order to select the optimal configuration. The analysis include different moments of the day, since the ambient temperature, the ground temperature and the temperature of the facilities changes along the day and, hence, the seeing degradation is different. Also, several conditions on thermal control have been taken into account to obtain an estimate of the convenience and effect of the thermal control mechanism. The final objective of the analysis was to keep the temperature of the surfaces of the facility as close as possible to the ambient temperature so as to minimize the local seeing effect. Once the temperature maps were obtained for the selected configurations, the CFD analyses were performed to evaluate the local seeing degradation.
Finally, an analysis of different alternatives for the EST telescope structure has been performed. The analyses have been centred on the Rocking-Chair, Gantry and Yoke models. The performance of each structural model has been evaluated according to their dynamic behaviour, as well as to the errors induced by gravity and the wind loads acting on the structure. An alternative configuration for the upper section of the telescope tube has also been considered, as well as the possibility of changing the current Nasmyth platform position from the right side of the telescope to the rear side.
All the analyses, simulations and the results are described in the corresponding reports.
sWP70.3.2 Development of an innovative heat rejecter prototype for GREGOR
Task 70.3.2 is devoted to the design and construction of a heat rejecter (HR) prototype for the GREGOR telescope. This HR follows the concept of the design proposed for EST, and, consequently, represents a proof of concept for the technical solution.
A thermal analysis was done to study the current contact sensors layout, evaluate the thermal environment at GREGOR F1 and estimate the possible improvement in the reduction of local seeing effects. A “cold” prototype was constructed to check the leakage, and the hydraulic and thermal performance. The Heat Rejecter (HR) final prototype has been produced and is ready for delivery to the GREGOR telescope. The HR front surface has been polished and coated. The device includes three interchangeable field diaphragms. Laboratory tests have demonstrated the good performance of the system. The final prototype has been realized after modifications deduced from the experience achieved with the first “cold” prototype to reduce the hydraulic friction resistance inside the HR body and pipes. A number of hydraulic tests have been performed on the final prototype. The heat rejecter will be integrated and tested at the telescope during the next maintenance period of the GREGOR telescope (foreseen for summer 2017).
WP80. Synoptic observations: Solar Physics Research Integrated Network Group (SPRING)
A network of telescopes with a small aperture but a large field-of-view can provide useful data to prepare observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures.
During the first period of the project, the scientific requirements were defined, from which the observational requirements were derived and translated into technical requirements. The translation into technical requirements followed the key idea to obtain a simple observatory, which is easy to operate and most cost effective. From the list of the observational requirements it is clear that a single instrument is not capable of fulfilling all those needs. Even if it can, its technical complexity would be too high, which finally would lead to high costs and low mean time between failures. For a network that should provide the data without interruption, this is not acceptable. Therefore the development had to follow the concept of a single platform that host several instruments.
Starting from the requirements for the spatial resolution, the detectors, instruments and front-end telescopes were defined. The technical feasibility of the proposed concept was extensively tested using the guiding telescope of the Vacuum Tower Telescope (VTT). Instead of a prefilter wheel a fast matrix filter shifter was used to obtain full-disk Dopplergrams in various wavelengths. Dual Fabry-Perot interferometers were operated in a collimated mount.
The overall technical concept is described in the corresponding document, including a study on the seeing conditions at the six possible sites Teide Observatory, Big Bear Observatory, Cero Tololo, Mauna Loa, Learmonth and Udaipur, estimates for data rates, and a description of the cloud monitoring.
In order to obtain a cost estimate, the combination of cameras, instruments and telescopes needed to be combined. Furthermore the cost for the building and the telescope tower were taken into consideration. The two concepts one with the fulldisk telescope (FDIS) and one with the multiplexed spectrograph (mxSPEC) result in a cost of up to 51.3 Mio. Euro for setting up a network with six nodes around the world.
Overall, the study was successfully completed. SPRING has gained large attention by the scientific community. The need for synoptic observations of the Sun is expressed by the world-wide research community. Plans for setting up a ground-based network of observatories can now be built on this technical and feasibility study.
WP90. Access Programme to key ground-based infrastructures (led by KIS)
The oversight of the TAS programme is described in the report on WP20. The ACCESS time at the SOLARNET telescopes and instruments was allocated and delivered successfully. The ACCESS programme enabled a large number of users to take advantage of state-of-the-art facilities. In particular it brought a large number of new users to the telescopes. It helped to foster small national European communities which cannot afford such large facilities.
All relevant number on the ACCESS programme are given in the document "Report on the TAS programme 2013 - 2016". A total of 466 observing days were allocated and executed. These are 15 days more than planned, but since the real unit costs turned to be slightly smaller than the initial estimated costs, the used budget amounts to almost the same amount.
Both PI and service mode campaigns have delivered valuable state-of-the-art data sets that led to scientific publications in refereed scientific journals.
WP100. Access to Science Data Centres. Space missions
The centers have all streamlined the access to the facilities and have significantly added new resources to the databases through the SOLARNET project. The three centers have increased awareness of the existence of the space-based data to the solar physics community in Europe. This has been accomplished through participation in conferences and meetings. More details about the individual centers are given below.
SubWorkpackage 101: HINODE/IRIS. Access to Science Data Centres (UiO)
Detailed monthly reports on the activity in the Science Data Centre have been produced followed by detailed usage statistics (complete data available from http://sdc.uio.no/sdc/webstats). Access to the Hinode Science Data Centre was provided throughout the whole project. The most significant improvement through the SOLARNET project was the addition of all data from the Interface Region Imaging Spectrograph (IRIS). The data was made available from the first day of open data policy, October 31st 2013. During the reporting period, all data from Hinode and IRIS have been continuously ingested into the Data Centre. Additional search possibilities have been added and the data presentation has been further improved through improved thumbnail pictures.
SubWorkpackage 102: BE-WISSDOM. Access to Science Data Centres (ROB)
The Solar Dynamics Observatory (SDO) was launched in January 2010 carrying three primary instruments the Atmospheric Imaging Assembly (AIA), the Helioseismic and Magnetic Imager (HMI) and the Extreme ultraviolet Variability Experiment (EVE). These instruments produce between 1 and 2 TB of data per day. The goal of the European SDO data centre is to be a powerful cache for the data produced by SDO. By doing this we diminish the load on the USA SDO data centre at the Joint Science Operations Center (JSOC) at Stanford university and provide the Solar community with a quick and reliable access to SDO data.
The hardware of the SDO data centre was updated in 2015. The SDO datacenter now has the following hardware: The storage array is made up of a SuperMicro storage server and a JBOD (just a bunch of disks) storage extension. These are equipped with 45 4TB hard disks and two 80 GB solid state drives, providing about 120 TB of usable storage after formatting. The database server was purchased with 80 GB and 800 GB SSD hard disks to store the indices of the databases, maximum of 12 SATA/SAS hot-swap hard disks, a redundant power supply and 256 GB double data rate type SDRAM.
SubWorkpackage 103: GSC-SDO. Access to Science Data Centres (MPS)
The German Science Center for the Solar Dynamics Observatory (GSC-SDO) is dedicated to the acquisition and distribution of data from the Helioseismic and Magnetic Imager (HMI) instrument that is onboard the Solar Dynamics Observatory (SDO). The GSC-SDO is hosted by the Max Planck Institute for Solar System Research (MPS) in Goettingen, Germany.
The ultimate goal of sWP103 is to make HMI data (and other relevant data) available to the European solar physics community through the Data Record Management System, NetDRMS. NetDRMS is a software suite developed by Stanford University specifically for managing SDO observations and sharing the data worldwide: http://jsoc.stanford.edu/netdrms . NetDRMS is fully operational at the GSC-SDO and data are automatically transferred directly from Stanford University as soon as they are available. The GSC-SDO has an online raw storage capacity of 888 TB dedicated to SDO/HMI data and other related data products.
The GSC-SDO is “subscribed" to the several data series from the HMI instrument. When a NetDRMS site subscribes to a particular data series, that site by default receives all the meta-data, i.e. keyword, value pairs, directly from Stanford University. There is a separate mechanism that must be triggered in order to also receive the images. Furthermore, this can be done selectively so you can choose which images you want in your local NetDRMS system. This latter feature is particularly useful if local storage resources are not sufficient to store all the images, allowing a site to only transfer the images of interest.
The MPS strongly promotes the GSC-SDO whenever possible. In particular, it is described in many posters from the MPS that are prepared for meetings/conferences worldwide. Many MPS publications also acknowledge and/or describe the GSC-SDO. This is a consequence of the GSC-SDO being a local facility for MPS scientists, and hence many of the scientific results coming from the MPS made use of data from the GSC-SDO.
Potential Impact:
IMPACT, DISSEMINATION ACTIVITIES AND EXPLOITATION RESULTS
The SOLARNET project has represented a major initiative for the field of high-resolution solar physics. The objectives of the project have been successfully achieved with a considerable impact in all areas aimed at the beginning. In the following paragraphs, this impact of SOLARNET in the most relevant areas is briefly described.
1.- Integration of the key research infrastructures
The most prestigious and powerful European facilities in the field have been offered to the whole community in a co-ordinated way. SOLARNET has organised the access to the German VTT and GREGOR, the Swedish SST, and the French THEMIS telescopes, as well as to the Italian IBIS and British ROSA instruments operating at the American DST telescope. This access has been managed by the EAST Time Allocation Committee (TAC), composed by one representative of the institutions that operate the telescopes/instruments handled by SOLARNET, plus one representative of IAC and NSO.
The EAST TAC granted observing time in PI mode as well as in the ‘service’ mode. The PI mode is the traditional observing mode, in which the principal investigator and parts of his/her team goes to the telescope and performs the observations. The technical support is typically limited to the telescope and the instruments need to be operated by the observing team. The service mode is a novel observing mode in solar physics and it is fundamental for an optimum operation of future facilities. In this mode the proposers describe the scientific objective and give a detailed explanation on what data they need. The data is acquired by a service team at the telescope following the requirements that the proposers have specified.
The EAST TAC announced calls for proposals for the observing seasons covered by the SOLARNET period. The calls were issued in ‘SOLAR NEWS’, which is subscribed by most Solar Physicists worldwide. Also, the EAST TAC informed all ‘known’ people that could be interested in the ACCESS program. In addition the Project office implemented a web page within the SOLARNET internet portal to promote the ACCESS program, and distributed a poster on the ACCESS program. The proposal deadlines were in mid-January of each observing season. The number of submitted proposal was higher than expected. The EAST TAC checked the proposals for eligibility and contacted the telescope operators for checking the technical feasibility. The EAST TAC selected two independent referees who rated the proposals according to their scientific merit. Based on the scientific merit, the EAST TAC granted the observing time.
The ACCESS programme was most successful in being oversubscribed and bringing new users to the telescopes. In the course of the 4-year program, the oversubscription rate increased steadily. In 2016 - the fourth year - the ACCESS program was oversubscribed by 200% (26 proposals for 13 allocated projects). If only PI mode campaigns are considered, the oversubscription rate was even higher. It amounted to 250% (22 proposals for 9 allocated projects).
During the whole project, the EAST TAC allocated a total of 68 observing campaigns with 466 observing days. 353 users from 18 countries were awarded with observing time and 59 received a travel & subsistence grant to perform the observations. 30 of those 59 were new users.
SOLARNET brought together all the data pipeline experts from the SOLARNET telescopes, as well as US data scientists that are involved in the data pipelines of the upcoming 4m solar telescope DKIST in Hawaii. Having brought together this community is a major success of the project. A main result of this joint activity is the generation of data pipelines for the major European solar instruments. At the start of SOLARNET, some of these instruments were only in a planning state, and for the instruments that did exist, the pipelines were in different stages of development. In general the existing pipelines were not user friendly, their data formats ad hoc, and the output seldom suitable for being ingested in virtual observatories. SOLARNET issued guidelines with general recommendations for those pipelines, so that external observers could easily get their data reduced to facilitate the posterior analysis. Thanks to SOLARNET, there are now modern pipelines for all major European solar instruments. The pipelines have been raised to a new standard where they are more reliable and easy to use. This has led to a higher productivity and more and better science. The work with conforming the pipelines to the needs of virtual observatories will allow the future development of such databases.
A number of prototypes have been designed, constructed and tested to improve the future capabilities of the facilities handled by SOLARNET, and having in mind the future large aperture European Solar Telescope (EST):
a.- A full study has been conducted to implement an adaptive optics system for the THEMIS telescope, taking into account the current technical state of the telescope and the scientific goals derived from the polarimetric usage of the long slit spectrograph attached to the telescope. The AO design has been done keeping valid the important specification property of this telescope as a "calibration-free" telescope.
b.- An innovative heat rejecter prototype for GREGOR has been designed and fabricated. This heat rejecter follows the concept of the design proposed for EST, and, consequently, represents a proof of concept for the technical solution proposed for this new large facility.
c.- A prototype of a large (150 mm in diameter), tunable Fabry-Perot Interferometer (FPI) has been built. The results point towards the feasibility of a 150 mm, capacity controlled FPI built using “conventional” methods, for use in solar imaging spectroscopy. This activity represents an important step towards the optimization of such devices for use in instrumentation tailored for large solar telescopes like EST.
d.- Two prototypes of integral field units to obtain the spectra of all points in a 2-D region simultaneously have been especially designed and constructed for the telescopes GREGOR and SST. One is based on an image slicer optimized for GRIS, the infrared spectrograph of the GREGOR telescope. The other is based on using microlenses to feed the spectrograph TRIPPEL of the SST. Both prototypes have been successfully tested at the corresponding telescopes and validate these concepts for the European Solar Telescope instrumentation.
e.- A fast solar polarimeter has been designed and tested using novel detector concepts, especially designed within SOLARNET, and a new polarizing beamsplitter.
2.- Contribution to a Solar Virtual Observatory
The standard for metadata and data archiving have been defined and adopted by the various SOLARNET telescopes and instruments. In the community this is known as the SOLARNET standard for metadata and is most relevant to the community world-wide. The team of the US 4m solar telescope DKIST who is designing the concept for their data pipelines is also taking advantage of 'our' European SOLARNET standard. An SVO prototype has been produced that allows the retrieval of ground-based and space-based data. A data archive exists for the GRIS instrument and science ready can be retrieved by external users. The GRIS instrument data pipelines provide the necessary metadata to be ingested into the SVO prototype, such that the GRIS data is fully searchable by external scientists. A similar data archive is provided by BBI@GREGOR, GFPI@GREGOR, and the Italian IBIS@DST data. Their metadata is also ingested into the SVO prototype. The prototype SVO can be accessed at http://solarnet.oma.be/.
3.- Contribution to the realisation of a future large European ground-based facility (EST)
All the aspects mentioned above (access to external observers, service mode, pipeline development, guides for data formats to accommodate virtual observatory rules, SVO prototype fed with high-resolution ground-based data, improvement of telescope capabilities and new concepts of instrumentation) are key aspects in which the impact of SOLARNET on the major European high-resolution facilities to observe the Sun has been made apparent. They are also crucial for the definition and operation of the European Solar Telescope (EST), so that this facility is optimally designed according to its scientific goals, operated in an efficient way and delivers data that is accessible to the whole community.
In addition to those achievements, specific works have been carried to optimise the design of EST. Specifically, SOLARNET has addressed the following tasks:
a.- A site-testing campaign has been carried put to characterise the turbulence above the two Canarian observatories (Observatorio del Teide, in Tenerife, and Observatorio del Roque de los Muchachos, in La Palma). Results have been produced showing the statistics and seeing properties at different heights from the ground and the daily variation for the different seasons of the year. This SOLARNET activity will be most important for the final decision of the location of EST.
b.- Specific simulations have been performed under different turbulent conditions to determine the optimum height for the multi-conjugate adaptive optics system (MCAO) deformable mirrors (DM). Using real atmospheric profiles from ORM and OT and the evaluated MCAO DMs, it has been shown that, after correction, a homogeneous Strehl higher than 40% can be achieved over a 1 arcmin FoV for all elevations. These numerical results have been validated with analytical equations and a complete error budget.
c.- Thermal analyses of the EST environment have been done for different configurations and weather conditions. These analyses have been used for specific CFD simulations to evaluate the seeing conditions and help the SOLARNET consortium to decide the optimum design of the EST facilities.
d.- A study of different alternatives for the EST telescope structure has been performed. The performance of each structural model has been evaluated according to their dynamic behaviour, as well as to the errors induced by gravity and the wind loads acting on the structure.
e.- SPRING, a network of telescopes with a small aperture but a large field-of-view has been defined and the technical solutions studied. This network is useful to provide data that can help the preparation of observing campaigns with large-aperture high-resolution telescopes and complement the data taken with them. Distributed in a world-wide network, these small apertures can represent an invaluable supporting tool for coordinated observations with the major infrastructures.
4.- Synergies with other communities
Four meetings were organised by SOLARNET to foster synergies between our high-resolution ground-based solar physics community with other fields of research. Specifically, these meetings addressed the topics: “Synergies between ground- and space-based solar research”, “Solar and stellar magnetic activity“, “The Sun, the stars and solar-stellar relations”, and “The physics of the Sun from the interior to the outer atmosphere”. The increasing impact of SOLARNET within the community is demonstrated by the fact that an increasing number of attendees participated in the SOLARNET meetings (60, 61, 84, and 143, respectively).
5.- Strengthening and reinforcement of the European Research Area
Modern science is entirely global, with information shared between researchers across the world virtually instantly. For astronomy and other similar disciplines there is another reason for globalisation - the high cost of the infrastructure that we need today. This means that countries must work together to plan for the future, pooling available resources across the whole of Europe.
SOLARNET has led to an increased mobilization and movement of human resources across Europe and their exposure to different ideas and methods, which are likely to lead to an improvement in career development prospects, particularly for the young researchers involved in the project. The adequate training of young researchers is fundamental is most important to create the atmosphere for international co-operation, spread knowledge and reinforce the idea of a European Research Area.
With that goal in mind, SOLARNET has carried out an ambitious Mobility Programme with seven calls for young researchers to allow them to begin early to establish international collaborations and reinforce the contacts between different groups. 20 young researchers benefitted from this programme, with an average stay of 8.3 weeks at another institution.
As a complement of the Mobility Programme, SOLARNET has organised five schools on fundamental research areas of solar physics: “Introduction to Solar Physics”, “Ground- and space-based inseruments”, “Solar magnetic fields: modelling and measuring techniques”, “Solar MHD and reconnection”, and “MHD waves and oscillations in the solar atmosphere”. The schools were accompanied with related Workshops that facilitated the interaction of the young researchers with well-established scientists and the presentation of their research work.
6.- Innovation and socio-economic impact
SOLARNET has contributed to reinforce significantly the relationship with industry for the most advanced technological challenges addressed by the project. This is the case of the partners Winlight Optics (for the fabrication of the imager slicer for 2D spectroscopy), SRS (for the fabrication of the heat rejecter prototype for the telescope GREGOR), PNSensor (for the development of high-speed low-noise new sensor concepts), and HANKOM (for its participation in the minimisation of the impact of local turbulence on image quality).
In addition to this direct participation in the most technical works, innovation aspects were channeled through the participation in the project of TECNALIA. This partner has a wide experience on this type of activities at EU level with industrial experts, and their European networks and contacts, on Technology Transfer. The particular achievements achieved can be summarized:
a.- Identification and analysis of technologies, techniques and new concepts related to solar physics telescopes and instruments
b.- Dissemination of the innovative aspects and findings of the project towards industry
c.- Organisation of workshops related to technology transfer
d.- Assessment of space technologies
e.- Dissemination of the project (specifically the technology offer and needs)
f.- Organisation of Common Exploitation Booster workshop
g.- Identification of future projects
h.- Promotion of synergies with other EU projects
7.- Dissemination activities
The main objectives of SOLARNET for the dissemination and exploitation of the knowledge derived from the project have been:
a.- make solar physics research more attractive and more comprehensible for a targeted audience.
b.- achieve visibility for the project and its findings across Europe and worldwide, both inside and outside the scientific community;
c.- promote the awareness of science.
d.- encourage talented students and scientists to join the partner institutes and enterprises;
e.- generate industrial interest for the developed products or services.
In order to spread this knowledge, the outreach and dissemination approach of the consortium has foreseen a well-defined set of targeted audiences at European level: the scientific community (with the presentation of SOLARNET activities results and the EST project to the main scientific meetings), industry (with the organisation of workshops related to technology transfer and presenting SOLARNET at a number of technological fora), policy makers (SOLARNET and EST have been continuously presented to the national funding agencies), media and general public (press releases and publicly available videos have been produced).
The European Strategic Forum for Research Infrastructures (ESFRI) has recognized the efforts done by the high-resolution Solar Physics community with the SOLARNET project and has included EST in the list of strategic European infrastructures (ESFRI roadmap) in March 2016. This is a major achievement that undoubtedly will foster the community to higher and more ambitious goals to make present telescopes more efficient, increase the scientific return of the data taken with the most advanced facilities, and make EST become a reality.
List of Websites:
http://www.solarnet-east.eu/
Contact details available at http://www.solarnet-east.eu/contact
