Periodic Reporting for period 2 - SWAMI (Space Weather Atmosphere Model and Indices)
Okres sprawozdawczy: 2019-01-01 do 2021-03-31
The SWAMI project aimed to provide an improved and comprehensive representation of the neutral atmosphere from the surface to 1500 km altitude. The three objectives of SWAMI were:
OB1. To develop a model of the whole atmosphere, MCM, with a science as well as operations-focused approach. Two existing models, the UM and the DTM, will be extended and blended to produce this unique new whole atmosphere model, which provide estimates of both climatology and space weather variability.
OB2. To provide new high-cadence geomagnetic Kp-indices, including its nowcast and predictions to be used in the UM and DTM. These products are equally useful for a wide range of space weather services that rely on rapid geomagnetic activity specification.
OB3. To develop steps, including provision of software, model output, or data sharing facilities, to transition the improved model system into operations.
The project ends with a high level of success despite difficulties in dissemination activities, in-person meetings, and recruitment due to the COVID-19 pandemic.
OB1. To develop a model of the whole atmosphere, MCM, with a science as well as operations-focused approach. Two existing models, the UM and the DTM, will be extended and blended to produce this unique new whole atmosphere model, which provide estimates of both climatology and space weather variability.
OB2. To provide new high-cadence geomagnetic Kp-indices, including its nowcast and predictions to be used in the UM and DTM. These products are equally useful for a wide range of space weather services that rely on rapid geomagnetic activity specification.
OB3. To develop steps, including provision of software, model output, or data sharing facilities, to transition the improved model system into operations.
The project ends with a high level of success despite difficulties in dissemination activities, in-person meetings, and recruitment due to the COVID-19 pandemic.
The UM upper boundary at 85 km was extended to 152 km. The UM radiation scheme was modified to be more accurate in the upper mesosphere and lower thermosphere. Non-local thermodynamical equilibrium heating was implemented to ensure more accurate radiative heating rates above around 70 km in altitude. The radiation scheme was also further developed to enable the calculation of photolysis rates at high spectral resolution. The coupling was not completed and instead a relaxation to climatological temperature was included to provide a simplified representation of chemical heating in the lower thermosphere. A molecular diffusion and viscosity scheme was also implemented in the UM but requires further development. Model instabilities were suppressed by a modification of the UM’s existing sponge layer. Comparisons of UM and meteor radar data in the 80-100 km region show mean winds and tides are often of good quality, but above it compares much less well.
The benchmark thermosphere model DTM2013 has been improved and modified. Due to unavailability or delays in provision of data, modifications to drivers, and third party requests, it was necessary to do this in stages resulting in several unpublished intermediate models. An operational model driven by the established F10.7 and Kp indices for solar and geomagnetic activity, respectively, and a more accurate research model, which uses the not yet operationally accredited indices F30 and the new hourly Hpo, were developed. Assessment of the research DTM2020 revealed that it is the least biased and most precise model compared to assimilated data, a significant improvement over DTM2013, and validated through comparison with independent density data. The operational DTM2020 is less accurate than the research model except at 800 km altitude. It has comparable or slightly higher precision than DTM2013, despite using F10.7 instead of F30 as solar activity driver.
The User survey "Existing and Improved Geomagnetic Indices" indicated that the preferred resolution for the Kp-like index and preferred forecast horizon for indices were 60 minutes and up to 72 hours, respectively. During the development of the new Kp-like index, the open-ended feature has been added. The new planetary high-cadence open-ended Hpo index family with 60- and 30-minutes resolution, including nowcast and forecast, are currently running in an operational service. Archive data of these indices back to 1995 is provided. The operational service providing Kp and Hp60 forecast up to 72 hours ahead is based on machine learning models. An archive of Kp forecast for the historical period 1998-2017 is also available.
DTM and UM models have been combined to create the whole atmosphere model MCM. A survey was carried out to prospective and likely users to define the requirements for the MCM software. In April 2021, MCM was released as a software library that can be easily integrated. MCM has been published on Github and runs at the Virtual Space Weather Modelling Centre. It has been provided and will be tested soon at ESA, CNES, JPL and NASA.
SWAMI FINAL ACHIEVEMENTS:
Two new DTM2020 models were developed: an operational model driven by the established F10.7 and Kp indices for solar and geomagnetic activity, respectively, and a more accurate research model, which uses the not yet operationally accredited indices F30 and the new hourly Hpo. The research DTM2020 is the least biased and most precise model compared to assimilated data, which is confirmed through comparison with independent data (SET HASDM).
A new feature unique to DTM is the uncertainty estimate of the density prediction.
The MCM model, which extends from the surface to approximately 1500 km, was created by combining UM results in the form of monthly-mean grids with the operational DTM2020 model. The MCM software library was released, which can be easily integrated in orbit software. MCM is available on Github and runs at the Virtual Space Weather Modelling Centre.
High resolution, open-ended geomagnetic disturbance index Hpo was created: a 30-min and 60-min index, based on the same data and algorithm used for the 3-hourly Kp index, and its forecast up to 72 hours. The Hpo index family reflects all geomagnetic disturbance levels and is not capped like Kp = 9. Major geomagnetic storms can thus better be identified, as well as the onset of storms due to the higher time resolution.
The benchmark thermosphere model DTM2013 has been improved and modified. Due to unavailability or delays in provision of data, modifications to drivers, and third party requests, it was necessary to do this in stages resulting in several unpublished intermediate models. An operational model driven by the established F10.7 and Kp indices for solar and geomagnetic activity, respectively, and a more accurate research model, which uses the not yet operationally accredited indices F30 and the new hourly Hpo, were developed. Assessment of the research DTM2020 revealed that it is the least biased and most precise model compared to assimilated data, a significant improvement over DTM2013, and validated through comparison with independent density data. The operational DTM2020 is less accurate than the research model except at 800 km altitude. It has comparable or slightly higher precision than DTM2013, despite using F10.7 instead of F30 as solar activity driver.
The User survey "Existing and Improved Geomagnetic Indices" indicated that the preferred resolution for the Kp-like index and preferred forecast horizon for indices were 60 minutes and up to 72 hours, respectively. During the development of the new Kp-like index, the open-ended feature has been added. The new planetary high-cadence open-ended Hpo index family with 60- and 30-minutes resolution, including nowcast and forecast, are currently running in an operational service. Archive data of these indices back to 1995 is provided. The operational service providing Kp and Hp60 forecast up to 72 hours ahead is based on machine learning models. An archive of Kp forecast for the historical period 1998-2017 is also available.
DTM and UM models have been combined to create the whole atmosphere model MCM. A survey was carried out to prospective and likely users to define the requirements for the MCM software. In April 2021, MCM was released as a software library that can be easily integrated. MCM has been published on Github and runs at the Virtual Space Weather Modelling Centre. It has been provided and will be tested soon at ESA, CNES, JPL and NASA.
SWAMI FINAL ACHIEVEMENTS:
Two new DTM2020 models were developed: an operational model driven by the established F10.7 and Kp indices for solar and geomagnetic activity, respectively, and a more accurate research model, which uses the not yet operationally accredited indices F30 and the new hourly Hpo. The research DTM2020 is the least biased and most precise model compared to assimilated data, which is confirmed through comparison with independent data (SET HASDM).
A new feature unique to DTM is the uncertainty estimate of the density prediction.
The MCM model, which extends from the surface to approximately 1500 km, was created by combining UM results in the form of monthly-mean grids with the operational DTM2020 model. The MCM software library was released, which can be easily integrated in orbit software. MCM is available on Github and runs at the Virtual Space Weather Modelling Centre.
High resolution, open-ended geomagnetic disturbance index Hpo was created: a 30-min and 60-min index, based on the same data and algorithm used for the 3-hourly Kp index, and its forecast up to 72 hours. The Hpo index family reflects all geomagnetic disturbance levels and is not capped like Kp = 9. Major geomagnetic storms can thus better be identified, as well as the onset of storms due to the higher time resolution.
PROGRESS BEYOND THE STATE OF THE ART
An innovative method was developed to calculate photolysis rates at high spectral resolution while retaining both the accuracy and computational efficiency of the radiative flux calculations. The UM radiative heating scheme was extended to include radiative heating from the EUV and FUV wavelengths. Since the UM radiation scheme is very flexible and can be used for a wide range of applications, this extension to the EUV and FUV means that the UM radiation scheme can now be used in the simulation of both the current-day thermosphere and a future thermosphere that has possibly been affected by climate change.
The DTM2020 models are based on the most accurate density data, which are 20-30% smaller than used in previous models. The lower densities have a significant impact on satellite lifetime estimations, which will be longer impeding the reentry rule within 25 years after the end of operations. The research model is the most accurate thanks to its drivers, the Hpo index and the F30 solar index. Only DTM2020 provides estimates of the uncertainty of its prediction, which was a specific user request. Taking into account the uncertainty of the air drag in the orbit propagation will make conjunction analysis more robust.
A new planetary open-ended Hpo index family is developed. Operational service of nowcast Hp60 and Hp30 plots and Hp60, ap60, Hp30 and ap30 data files is installed, and archive data go back to 1995. Several machine learning models of Kp forecast and Hp60 forecast have been developed and tested on historical data. An archive of Kp forecast for the historical period 1998-2017 is available. Based on the machine learning models above mentioned, a real-time operational service providing Kp and Hp60 forecast up to 72hours ahead has been established using real-time solar wind data as input.
An innovative method was developed to calculate photolysis rates at high spectral resolution while retaining both the accuracy and computational efficiency of the radiative flux calculations. The UM radiative heating scheme was extended to include radiative heating from the EUV and FUV wavelengths. Since the UM radiation scheme is very flexible and can be used for a wide range of applications, this extension to the EUV and FUV means that the UM radiation scheme can now be used in the simulation of both the current-day thermosphere and a future thermosphere that has possibly been affected by climate change.
The DTM2020 models are based on the most accurate density data, which are 20-30% smaller than used in previous models. The lower densities have a significant impact on satellite lifetime estimations, which will be longer impeding the reentry rule within 25 years after the end of operations. The research model is the most accurate thanks to its drivers, the Hpo index and the F30 solar index. Only DTM2020 provides estimates of the uncertainty of its prediction, which was a specific user request. Taking into account the uncertainty of the air drag in the orbit propagation will make conjunction analysis more robust.
A new planetary open-ended Hpo index family is developed. Operational service of nowcast Hp60 and Hp30 plots and Hp60, ap60, Hp30 and ap30 data files is installed, and archive data go back to 1995. Several machine learning models of Kp forecast and Hp60 forecast have been developed and tested on historical data. An archive of Kp forecast for the historical period 1998-2017 is available. Based on the machine learning models above mentioned, a real-time operational service providing Kp and Hp60 forecast up to 72hours ahead has been established using real-time solar wind data as input.