Final Report Summary - SPWRT (Spacewire RT)
The SPACEWIRE-RT project has created a spacecraft on-board networking technology that is able to fulfil many on-board communication requirements including payload data-handling and avionic applications. In addition the project has substantially strengthened collaborative links between Russian and European academia and aerospace industry, building on the strengths and capabilities of the project partners.
The requirements for spacecraft on-board communications were researched by the industrial partners covering both Russian and European requirements and a comprehensive set of use cases were defined. The SpaceFibre network technology being developed by the European Space Agency was identified as a good baseline for SPACEWIRE-RT. Enhancements to SpaceFibre necessary to meet the SPACEWIRE-RT requirements were identified and research undertaken in the areas of quality of service (QoS), fault detection, isolation and recovery (FDIR) and network layer concepts. The design for a simple, powerful and comprehensive quality of service mechanism resulted. This QoS mechanism was then extended to include specific classes of fault detection, 'babbling idiot' and quiet unit detection, in support of FDIR. Specifications for the SpaceFibre quality layer, covering QoS and FDIR and the network layer were written.
The entire SpaceFibre protocol stack was validated through simulated using specification and description language (SDL) and SystemC. The results of these detailed simulations were used to help remove errors and inconsistencies in the specification and to make necessary clarifications and improvements to the SpaceFibre standard. Several problems with the SpaceFibre specification were identified and potential solutions were explored and traded-off. The specification was updated accordingly.
A very-high-speed integrated circuits (VHSIC) hardware description language (VHDL) implementation of the SpaceFibre interface was designed and implemented in a field programmable gate array (FPGA). This provided a means of further testing and validating the SpaceFibre specification including its QoS and FDIR features. Results of this implementation fed back into the specification and further inconsistencies and problems were resolved, with the experimental FPGA implementation being updated and re-tested. The feasibility of ASIC implementation of the SpaceFibre was explored and a complete SpaceFibre interface was shown to be only three or four times the chip area of a SpaceWire interface.
The SpaceFibre specification was updated several times throughout the SPACEWIRE-RT project and a Russian translation made. It is planned for this specification to be taken through formal standardisation with the European Cooperation for Space Standardisation (ECSS) once other aspects of SpaceFibre have been completed, including multi-lane operation. Research and development related to SPACEWIRE-RT is being carried forward by the project partners with the aim of spaceflight ready components and system within the next two to three years.
Project context and objectives:
Context
The need for standard network technology for spacecraft applications
The trend towards 'Operationally responsive space', where spacecraft can be rapidly assembled, configured and deployed, to meet specific mission needs, e.g. disaster support, requires flexible, high-performance, on board communication networks with plug-and-play capability. Earth observation missions employing synthetic aperture radar or hyperspectral imaging have very high data rates requiring multi-gigabit/s onboard network technology. The growing autonomy of scientific missions to remote planets requires networks that are robust and durable, able to recover from transitory errors and faults automatically. The importance of spacecraft mass reduction motivates the sharing of networks for payload data-handling and avionics. Avionics and robotics applications impose requirements on network responsiveness and determinism. Increasing international collaboration on scientific and Earth observation spacecraft requires standard network technology where a component developed by one nation will interoperate effectively with equipment developed by another. SPACEWIRE-RT fulfils these demanding requirements with a flexible, robust, responsive, deterministic and durable standard network technology that is able to support both avionics and payload data-handling applications.
SpaceWire
The existing SpaceWire technology [1],[2],[3] was a very successful first step in this direction, providing networking technology for payload data-handling on over 40 major space missions. It falls short, however, of the deterministic and galvanic isolation requirements for avionics systems. SpaceWire is also of insufficient data-rate for some space applications.
SpaceFibre
SpaceFibre [4],[5],[6],[7] is a very high-speed serial data-link currently being developed by ESA which is intended for use in data-handling networks for high data-rate payloads. SpaceFibre is able to operate over fibre optic and electrical cable and support data rates of 2 Gbit/s in the near future and up to 20 Gbit/s long-term. It aims to complement the capabilities of the widely used SpaceWire onboard networking standard: improving the data rate by a factor of 10, reducing the cable mass by a factor of four and providing galvanic isolation. SpaceFibre aims to support QoS along with FDIR. An important feature of SpaceFibre is that it has the same packet format as SpaceWire, so that several SpaceWire links can be easily multiplexed over a single SpaceFibre link and application software designed for SpaceWire can operate over a SpaceFibre network. SpaceFibre is currently missing the important QoS and FDIR capabilities and the network layer of SpaceFibre has yet to be defined.
Project objectives
The SPACEWIRE-RT research programme aims to conceive and create communications network technology, suitable for a wide range of demanding space applications where responsiveness, determinism, robustness and durability are fundamental requirements. This is a critical component technology for future spacecraft avionics and payload data-handling. The SPACEWIRE-RT project substantially strengthened collaborative bonds between the Russian and European organisations involved in the research and led to technology of vital importance for future space missions.
The SPACEWIRE-RT research programme had the following key objectives:
1. a justified set of requirements and use cases for SPACEWIRE-RT covering both spacecraft payload and avionics, which takes into account requirements from European and Russian spacecraft primes and equipment manufacturers;
2. a conceptual design and outline specification for SPACEWIRE-RT that fulfils the requirements and use cases;
3. validation of the SPACEWIRE-RT specification, with important, novel features of SPACEWIRE-RT networks tested using SDL and System-C models;
4. a SPACEWIRE-RT VHDL intellectual property (IP) core tested and validated in an FPGA implementation;
5. an assessment of the feasibility of implementing SPACEWIRE-RT as an ASIC core considering available ASIC technologies suitable for space;
6. a draft standard document for SPACEWIRE-RT in both English and Russian;
7. dissemination of the results of the SPACEWIRE-RT study to the European and Russian space industries and to the international space community;
8. an exploitation plan covering the availability of flight grade chips, test and development equipment, IP cores and a programme for a test flight.
Project team
A highly experience team of Russian and European academic and industrial organisations was assembled for the SPACEWIRE-RT project. The project team is introduced below:
University of Dundee (UnivDun) has long experience in spacecraft onboard network technology, writing the SpaceWire standard with support from the European Space Agency (ESA) and input from engineers across Europe. UnivDun has extensive experience in related IP core design having designed SpaceWire interface, remote memory access protocol (RMAP) and router cores for ESA, JAXA and other organisations. The principal responsibilities of UnivDun were overall project management, conceptual design and specification, VHDL IP core design and SPACEWIRE-RT standard specification.
Astrium GmbH is a major international spacecraft manufacturer that undertakes system integration and also designs and manufactures spacecraft payloads. Astrium was responsible with Submicron for the requirements and use cases for SPACEWIRE-RT based on their extensive spacecraft design experience.
Submicron is a Russian organisation that specialises in the design and production of spacecraft avionics. Submicron was responsible with Astrium for the requirements and use cases for SPACEWIRE-RT based on their extensive experience of spaceflight electronic equipment.
St Petersburg University of Aerospace Instrumentation (SUAI) are experienced in the simulation and testing of network technologies using SDL particularly in the field of aerospace. SUAI designed their own SpaceWire IP cores and devised an innovative interrupt mechanism for SpaceWire. SUAI was primarily responsible for the simulation and validation of the SPACEWIRE-RT protocols and for translating the SPACEWIRE-RT standard specification in to Russian. SUAI also contributed to the evolution of the SPACEWIRE-RT specification.
Elvees is a leading Russian ASIC design house with expertise in space grade electronics and chip design. Elvees was responsible for reviewing the SPACEWIRE-RT specification and considering the feasibility of ASIC implementation.
Project Results:
Project overview
The technical work began with work package one (WP1) and WP2. WP1 (Spacecraft avionics and payload use cases) focussed on the requirements and use cases for spacecraft onboard communication networks for both payload applications and avionics. Requirements for avionics networks were gathered, covering reliability, fault tolerance, fault isolation, performance, responsiveness and determinism. A series of use cases was explored covering a diverse set of avionics and payload applications. These requirements formed the basis for the technical capabilities of the SPACEWIRE-RT network technology.
WP2 (Concept and specification) reviewed existing networking technology, including commercial technologies, SpaceWire, SpaceFibre and other onboard technologies. From the requirements of WP1, a set of evaluation criteria were derived and the candidate networking concepts traded-off against one another. SpaceFibre was selected as the baseline technology for SPACEWIRE-RT. Several areas of SpaceFibre which required substantial further research and development were identified for investigation in the SPACEWIRE-RT project, including QoS, FDIR and the network layer. Concepts were devised that met the requirements from WP1 and a specification produced.
In WP3 (Simulation and validation), simulation models covering key aspects of SPACEWIRE-RT were designed and used to evaluate the proposed SPACEWIRE-RT solution. The simulation models were developed in SDL and used to run various validation scenarios. The results of the simulation were used to update the SPACEWIRE-RT specification and to inform the VHDL IP core development (WP4) and ASIC feasibility (WP5) activities.
WP4 (VHDL IP core prototype) began with the architectural design of the SPACEWIRE-RT IP core based on information from the outline SPACEWIRE-RT specification (WP2). Once the architectural design was ready the design and development of the VHDL IP core and associated test bench was carried out, followed by testing. The IP core was implemented in an FPGA and tested extensively. The results were presented to the SpaceWire working group and the SPACEWIRE-RT specification updated.
WP5 (ASIC feasibility) covered the review and evaluation of suitable ASIC technologies for SPACEWIRE-RT implementation, the procurement of suitable ASIC libraries and initial design and simulation work. Owing to the expense of ASICs, device manufacture was beyond the scope of the present activity, however, the principal risk areas with an ASIC development were investigated and the feasibility of ASIC implementation assessed.
The results from all the previous WPs were fed into WP6 (Standard draft) where a draft standard for SPACEWIRE-RT was written by UnivDun with all partners involved in review of the document. The SPACEWIRE-RT standard built on the SpaceFibre specification, adding in the quality and network layers. The draft standard was presented to the SpaceWire working group and feedback used to improve the document. SUAI translated the standard into Russian.
WP1: Spacecraft avionics and payload use cases
Astrium GmbH and Submicron gathered requirements from spacecraft manufacturers and spacecraft equipment suppliers across Europe and Russia respectively. These requirements covered many aspects of payload data-handling and avionics networks.
An important aspect of this work on requirements was the need for a range of QoS depending on the particular application. These include the capability to reserve network bandwidth to prevent different data flows interfering with one another, deterministic data delivery for control applications and high priority, very low-latency communication for time-synchronisation and side-band signalling.
The requirements were analysed by University of Dundee and compared against the characteristics of the planned SpaceFibre standard. It was clear that SpaceFibre would meet the majority of the requirements for SPACEWIRE-RT and so was adopted as the baseline network technology for very high data-rate applications. The outcome of this analysis was used to inform the work on QoS within SpaceFibre and resulted in a coherent precedence concept being proposed for QoS in SpaceFibre.
WP2: Concepts and specification development
SPACEWIRE-RT and SpaceFibre
The SPACEWIRE-RT project built on the emerging SpaceFibre standard which is being designed by the University of Dundee for ESA with inputs from international spacecraft engineers. The aim is to publish the final standard though the ECSS. The SPACEWIRE-RT project has contributed to the following parts of the SpaceFibre standard:
1. QoS mechanisms in the quality layer;
2. FDIR mechanisms in the quality layer;
3. network level concepts in the network layer.
The QoS and FDIR work has been fully integrated in the SpaceFibre standard specification. A baseline network layer specification has been added to the SpaceFibre standard specification. The overall SpaceFibre protocol stack has been rationalised and the SpaceFibre standard restructured accordingly.
Several version of the SpaceFibre specification were written as the SPACEWIRE-RT project progressed, incorporating QoS, FDIR and network layer specifications. Extensive simulation of each of these specification versions was carried out in the SPACEWIRE-RT project to help improve and validate the SpaceFibre draft standard. Draft E1 of the standard [7] was used for the critical simulation of the Virtual Channel and Retry Layers. Draft D [6] was used for the lower layers because this simulation work was carried out before draft E1 of the SpaceFibre standard was written. The results of the simulations were used to improve the SpaceFibre specification.
SpaceFibre protocol stack
The network layer is responsible for the transfer of application information over a SpaceFibre network. It provides two services: packet transfer service and broadcast message service. The packet transfer service transfers SpaceFibre packets over the SpaceFibre network, using the same packet format and routing concepts as SpaceWire uses. SpaceFibre supports both path and logical addressing. The broadcast message service is responsible for broadcasting short messages (8 bytes) to all nodes on the network. These messages can carry time and synchronisation signals and be used to signal the occurrence of various events on the network.
The management layer is responsible for configuring, controlling and monitoring the status of all the layers in the SpaceFibre protocol stack. For example, it can configure the QoS settings of the virtual channels in the QoS and FDIR layer.
The quality layer is responsible for providing quality of service and managing the flow of information over a SpaceFibre link. It frames the information to be sent over the link to support QoS and scrambles the packet data to reduce electromagnetic emissions. The QoS and FDIR layer also provides a retry capability, detecting any frames or control codes that go missing or arrive containing errors and resending them. With this inbuilt retry mechanism SpaceFibre is very resilient to transient errors.
The multi-lane layer is responsible for operating several SpaceFibre lanes in parallel to provide higher data throughput. In the event of a lane failing the multi-lane layer provides support for graceful degradation, automatically spreading the traffic over the remaining working links.
The lane layer is responsible for lane initialisation and error detection. In the event of an error the lane is automatically re-initialised. The lane layer encodes data into symbols for transmission using 8B/10B encoding and decodes these symbols in the receiver. 8B/10B codes are direct current (DC) balanced supporting alternating current (AC) coupling of SpaceFibre interfaces.
The physical layer is responsible for serialising the 8B / 10B symbols and for sending them over the physical medium. In the receiver the Physical layer recovers the clock and data from the serial bit stream, determines the symbol boundaries and recovers the 8B / 10B symbols. Both electrical cables and fibre-optic cables are supported by SpaceFibre.
The specific results of the SPACEWIRE-RT project that contributed to the SpaceFibre standard specification will now be described.
SpaceFibre quality of service
The specification, validation, prototype implementation and testing of the SpaceFibre quality layer was a major output of the SPACEWIRE-RT project. In this section, a detailed overview of this technology is provided.
Frames and virtual channels
To provide quality of service, it is necessary to be able to interleave different data flows over a data link or network. If a large packet is being sent with low priority and a higher priority one requests to be sent, it must be possible to suspend sending the low priority one and start sending the higher priority packet. To facilitate this SpaceWire packets are chopped up into smaller data units called frames. When the high priority packet requests to be sent, the current frame of the low priority packet is allowed to complete transmission and then the frames of the high priority packet are sent. When all the frames of the high priority packet have been sent, the remaining frames of the low priority packet can be sent.
Each frame has to be identified as belonging to a particular data flow so that the stream of packets can be reconstructed at the other end of the link. Low priority packets belong to one data stream and high priority packets belong to another data stream.
Each independent data stream allowed to flow over a data link is referred to as a virtual channel (VC). Virtual channels are unidirectional and have a QoS attribute, e.g. priority. At each end of a virtual channel is a virtual channel buffer (VCB), which buffers the data from and to the application. An output VCB takes data from the application and buffers it prior to sending it across the data link. An input VCB receives data from the data link and buffers it prior to passing it to the receiving application.
There can be several output virtual channels connected to a single data link, which compete for sending information over the link. A medium access controller determines which output virtual channel is allowed to send the next data frame. When an output VCB has a frame of data ready to send and the corresponding input VCB at the other end of the link has room for a full data frame, the output VCB requests the medium access controller to send a frame. The medium access controller arbitrates between all the output VCBs requesting to send a frame. It uses the QoS attribute of each of the requesting VCBs to determine which one will be allowed to send the next data frame.
Priority is one example of a QoS attribute. Other types of QoS are considered in the subsequent sections.
Precedence
For the medium access controller to be able to compare QoS attributes from different output VCBs, it is essential that they are all using a common measure that can be compared. The name given to this measure is precedence. The competing output VCB with the highest precedence will be allowed to send the next frame.
bandwidth reservation
When connecting an instrument via a network to a mass memory, what the systems engineer needs to know is 'how much bandwidth do I have to transfer data from the instrument to the mass memory?' Once the network bandwidth allocated to a particular instrument has been specified, it should not be possible for another instrument to impose on the bandwidth allocated to that instrument. A priority mechanism is not suitable for this application. If an instrument with high priority has data to send, it will hog the network until all its data has been sent. What is needed is a mechanism that allows bandwidth to be reserved for a particular instrument.
Bandwidth reservation calculates the bandwidth used by a particular virtual channel and compares this to the bandwidth reserved for that virtual channel to calculate the precedence for that virtual channel. If the virtual channel has not used much reserved bandwidth recently, it will have a high precedence. When a data frame is sent by this virtual channel, its precedence will drop. Its precedence will increase again over a period of time. If a virtual channel has used more than its reserved bandwidth recently, it will have a low precedence.
A virtual channel specifies a portion of overall link bandwidth that it wishes to reserve and expects to use, i.e. its expected bandwidth.
When a frame of data is send by any virtual channel, each virtual channel computes the amount of bandwidth that it would have been permitted to send in the time interval that the last frame was sent. This is known as the bandwidth allocation.
Each virtual channel can use this to determine its bandwidth credit, which is effectively the amount of data it can send and still remain within its expected bandwidth. Bandwidth credit is the bandwidth allowance less the bandwidth used accumulated over time.
The bandwidth credit is updated every time a data frame for any virtual channel has been sent. A bandwidth credit value close to zero indicates nominal use of bandwidth by the virtual channel. A negative value indicates that the virtual channel is using more than its expected amount of link bandwidth. A positive value indicates that the virtual channel is using less than its expected amount of link bandwidth.
To simplify the hardware required to calculate the bandwidth credit it is allowed to saturate at plus or minus a bandwidth credit limit, i.e. if the bandwidth credit reaches a bandwidth credit limit it is set to the value of the bandwidth credit limit.
When the bandwidth credit for a virtual channel reaches the negative bandwidth credit limit it indicates that the virtual channel is using more bandwidth than expected. This may be recorded in a status register and used to indicate a possible error condition. A network management application is able to use this information to check correct utilisation of link bandwidth by its various virtual channels.
For a virtual channel supporting bandwidth reserved QoS, the value of the bandwidth counter provides the precedence value for that virtual channel.
If the bandwidth credit counter reaches the minimum possible bandwidth credit value, it indicates that it is using more bandwidth than expected and a possible error may be flagged. This condition may be used to stop the VC sending any more data until it recovers some bandwidth credit, to help with 'babbling idiot' protection.
Similarly if the bandwidth credit counter stays at the maximum possible bandwidth credit value for a relatively long period of time, the VC is using less bandwidth than expected and this condition can be flagged to indicate a possible error.
The bandwidth credit value is the precedence used by the medium access controller to determine which VC is permitted to send the next data frame.
Priority
The second type of QoS provided by VCs is priority. Each VC is assigned a priority value and the VC with the highest priority (lowest priority number) is allowed to send the next data frame as soon as it is ready.
Within any level there can be any number of VCs which compete amongst themselves based on their bandwidth credit. A higher priority VC will always have precedence over a lower priority VC unless its bandwidth credit has reached the minimum credit limit in which case it is no longer allowed to send any more data frames. This prevents a high priority VC from consuming all the link bandwidth if it fails and starts babbling. More than one VC can be set to the same priority level in which case those VCs will compete for medium access using bandwidth reservation.
Scheduled
To provide fully deterministic data delivery it is necessary for the QoS mechanism to ensure that data from specific virtual channels can be sent (and delivered) at particular times. This can be done by chopping time into a series of time-slots, during which a particular VC is permitted to send data frames.
Each VC is allocated one or more time-slots in which it is permitted to send data frames. VC1 is scheduled to send in time-slot one and VC2 is scheduled to send in time-slots two and six. The time-slot duration is a system level parameter, typically 100 µs and there are 256 time-slots.
During a time-slot, if the VC is scheduled to send in that time-slot, it will compete with other VCs also scheduled to send in that time-slot based on precedence (priority and bandwidth credit). A fully deterministic system would have one VC allowed to send in each time-slot.
The schedule is always operating. If a user does not want to use scheduling the schedule table is simply filled completely, allowing any VC to send in any time-slot, competing with precedence.
Scheduling can waste bandwidth if only one VC is allowed to send in a time-slot and that VC is not ready. To avoid this situation, the critical VC can be allocated a time-slot and given high priority and another VC can be allocated the same time-slot with lower priority. In this way when that time-slot arrives the high priority VC will be allowed to send its data, but if it is not ready the VC with lower priority can send some data.
Time-slots can be defined by broadcasting start of time-slot signals, or by broadcasting time and having a local time counter which determines the start and end of each time-slot. The SpaceFibre broadcast message mechanism supports both synchronisation and time distribution.
The SpaceFibre QoS mechanism is simple and efficient to implement and it provides bandwidth reservation, priority and scheduling integrated together, not as separate options. Furthermore SpaceFibre QoS provides a means for detecting 'babbling idiots' and for detecting nodes that have ceased sending data when they are expected to be sending information.
SpaceFibre FDIR
The FDIR mechanism for SpaceFibre is another significant output of the SPACEWIRE-RT project. A brief overview of the FDIR for SpaceFibre is provided in this section.
SpaceFibre provides automatic fault detection, isolation and recovery. When a fault occurs on a SpaceFibre link, it is detected and the erroneous or missing information resent. SpaceFibre recovers from intermittent faults very rapidly, detecting faults, recovering and resending data faster than SpaceWire disconnects and reconnects a link. The retry mechanism does not depend on time-outs, naturally adapting to different cable delays.
Fault detection is provided by checking each 8B / 10B symbol for disparity errors and invalid 8B / 10B codes. SpaceFibre has selected the 8B / 10B K-codes it uses to have enhanced Hamming distance from data-codes. This means that a single bit error occurring in a data-code cannot result in a valid K-code used by SpaceFibre. In addition each data frame, broadcast frame, FCT, ACK and NACK are protected by a CRC.
Fault isolation is provided at various levels in SpaceFibre. AC coupling is used in the physical layer to prevent damage from faults that cause DC voltages exceeding the maximum permitted to appear on the transmitter outputs or receiver inputs. This feature also enables galvanic isolation to be implemented readily. At the quality level SpaceFibre provides time containment, containing errors in the data frame in which they occur and bandwidth containment, containing errors to the virtual channel in which they occur; an error in one VC does not affect data flowing in another VC. Babbling idiots are contained using the QoS mechanism described above.
Fault recovery is provided at the link level using a retry mechanism that resends data frames, broadcast frames and FCTs. The retry is very fast, uses a minimum amount of buffer memory and adapts automatically to different link lengths. In addition to the retry mechanism the multi-lane functionality includes graceful degradation on lane failure. If a lane fails permanently, so that a retry or re-initialisation does not recover lane operation, a multi-lane system will continue using the remaining lanes available. This reduces the bandwidth available but does not stop the link operating. For critical operations an extra lane can be included and the graceful degradation will then provide automatic replacement of a faulty lane. The bit error rate (BER) of a lane is monitored and a lane reported as faulty if the BER is above a level which results in the effective link bandwidth being unusable. This feature allows lanes that can re-initialise successfully but which will not run for very long before having to re-initialise again, to be detected, isolated and replace by a fully functional lane.
SpaceFibre networks
The third substantial contribution of the SPACEWIRE-RT project is a baseline concept for the SpaceFibre network layer. Initial validation of the network layer has been carried out by SUAI using SystemC.
A SpaceFibre network uses similar packet formats, packet addressing and routing concepts as SpaceWire. The main difference is that SpaceFibre includes virtual channels.
The SpaceFibre router comprises a number of SpaceFibre interfaces and a routing switch matrix. Each SpaceFibre interface has several virtual channels. The VC number for each virtual channel can be configured, except for VC0 which is a virtual channel used for configuration, control and monitoring of the SpaceFibre network. When a packet arrives on a SpaceFibre interface it is placed in the appropriate virtual channel, i.e. the one with the same VC number as it was transmitted on. The leading data character of the packet determines which port of the routing switch the packet is to be forwarded through using either path or logical addressing. The port that it is to be switched to must have a VC configured with the same number as the VC that the packet arrived on. The packet is then passed through the routing switch matrix and placed frame by frame in the VC of the output port. The packet is then transferred across the SpaceFibre link, competing with other VCs in that port for access to the link medium according to their precedence.
If a packet arrives and the output port that the packet is to be switched to does not have a VC with the same number as that on which it arrived, the packet is spilt and an error recorded.
Virtual channels can be used to construct virtual networks where a single VC number is used for connecting to all or several of the nodes attached to the network. Virtual channels can also be used to construct virtual point to point links from one node to another.
Physical layer
SpaceFibre is a very high-speed communication protocol. The QoS and FDIR protocols developed for SpaceFibre can potentially be applied to a range of slower speed network technologies. If a consistent set of QoS and FDIR protocols can be applied across network technologies like SpaceWire then the concepts will be common and application software can be readily reused regardless of the performance required from the network. SPACEWIRE-RT aims to use the QoS and FDIR protocols being developed for SpaceFibre across a range of different lower level protocol layers to cover a range of data rates and applications.
At present two physical layer protocols are being considered:
1. SpaceFibre current mode logic (CML):
The SpaceFibre protocol is designed to run at multiple Gbits/s using CML over copper cables for distances up to 5 m or fibre optic cables for distances of 100 m or more. Multiple lanes can be operated in parallel to increase the data rate. SpaceFibre uses 8B / 10B encoding and a phase locked loop (PLL), or similar technology, to recover a clock from the received data stream and to perform bit synchronisation.
2. SpaceFibre LVDS:
The SpaceFibre protocol could also be run at slower speed using low voltage differential signalling (LVDS), which is capable of 600 Mbits/s or possibly up to 1 Gbits/s data rates. LVDS is used in SpaceWire and therefore has space heritage. SpaceFibre LVDS would use 8B/10B encoding and a PLL for bit synchronisation.
It is proposed that both the CML and LVDS options for SpaceFibre are included in the SpaceFibre standard specification.
Oversampled SpaceFibre
SpaceFibre requires clock-data recovery circuitry in the receiver to recover the clock from the received data bit-stream. Normally, this is accomplished using some form of phase-locked loop (PLL) which requires specialised circuitry. Current space qualified FPGAs do not provide suitable PLL clock recovery circuitry, so an alternative clock recovery scheme is required if SPACEWIRE-RT is to be implemented in space qualified FPGAs. The receive signal can be over sampled (sampled with a clock of significantly higher frequency than the receive bit rate) and bit synchronisation achieved [8]. Data rates of up to 100 Mbits/s should be achievable, which is adequate for many spacecraft applications. This oversampling technique would simply be an alternative implementation for the clock-data recovery for SpaceFibre and be suitable at relatively low data-rates (e.g. 100 Mbits/s or slower).
WP3: SPACEWIRE-RT validation and simulation
One of the main objectives of the WP3 was to check the correctness and consistency of the SPACEWIRE-RT specification. This was done by specification and simulation using SDL for the lower layers of SpaceFibre and by simulation using SystemC for the network layer. Simulation and investigation was split into two activities: checking correctness and consistency of SPACEWIRE-RT and testing against requirements. From the start of WP3, three drafts of the SpaceFibre specification were released which were taken as the basis for the SPACEWIRE-RT standard. During specification and simulation several documents with proposals, change requests and issues for discussion were produced and provided to the University of Dundee. Overall, SUAI produced 139 change requests, 87 of which resulted in changes to the SpaceFibre standard. In this way the results of specification and simulation in SDL and SystemC were applied during the SPACEWIRE-RT development.
In the scope of testing SPACEWIRE-RT against the requirements of industry, a number of tests were performed, which evaluated and checked SPACEWIRE-RT functionality in point-to-point connection (primarily SDL model) and in networks with different topologies (SystemC model). To test basic mechanisms of the SPACEWIRE-RT two different network configurations were used: point-to-point configuration and mixed configuration (tree and circular configurations combined in one network). Overall this testing showed that SPACEWIRE-RT satisfies the requirements provided by the industry.
SDL point-to-point model
The SDL model was primarily focussed on checking internal functionality and how all mechanisms of all layers work in common in one node. The SDL model was layered according to the SpaceFibre specification and service access points (SAPs) were specified as the interfaces between layers.
SystemC network model
SystemC network models were primarily focussed on simulation and checking of network aspects of the SPACEWIRE-RT, performance parameters (e.g. latency) and some features that cannot be covered by SDL point-to-point simulation (e.g. broadcast distribution, QoS, etc.).
WP4: SPACEWIRE-RT VHDL IP core development
SpaceFibre VHDL IP core
The QoS and FDIR capabilities for SpaceFibre were implemented in an experimental SpaceFibre VHDL IP core so that the concepts could be tested in hardware. The design was implemented in a STAR Fire unit from STAR-Dundee, which contained a suitable FPGA with SpaceWire and SpaceFibre connectors attached to the FPGA. The design was extensively tested using the inbuilt SpaceFibre link analysis capabilities of the STAR Fire unit.
Following testing of the hardware implementation, improvements were made to the SpaceFibre specification and the SpaceFibre VHDL IP core was updated and re-tested. The design, simulation, implementation, test and update to the specification cycle were iterated several times as problems were discovered, solutions devised and the specification polished. This cycle of improvement ran concurrently with the improvements resulting from the simulation of the standard specification in WP3. The black cables plugged into the front panel are the SpaceFibre electrical cables. The FPGA is the largest chip on the right hand side of the unit.
Oversample SpaceFibre VHDL IP core
An oversampling technique suitable for SPACEWIRE-RT based on [8] was designed, simulated, implemented and tested in an FPGA. SpaceFibre ran successfully in the FPGA using oversampling for clock-data recovery.
The design was implemented in a pair of STAR-Dundee STAR Fire units. The two STAR Fire units were modified so that each unit had one normal SpaceWire interface (Port 1) and one oversampled SpaceFibre interface (port 2). A host PC provided a stream of SpaceWire packets via a SpaceWire USB Brick which were sent to the SpaceWire interface on one STAR Fire unit. The SpaceWire packets were then sent out of the Oversampled SpaceFibre interface to the other STAR Fire unit. The SpaceWire packets were received and looped-back using a SpaceWire router inside the STAR Fire unit so that they were sent back out of the Oversampled SpaceFibre interface. The SpaceWire packets were received back at the first STAR Fire unit and passed up to the host PC for checking, via the SpaceWire USB Brick.
Software running on the host PC checked the SpaceWire packets received noting any errors. The 'Sink Statistics' dialog box displays the total data packets received, data rate statistics and any detected errors. This shows that packets are successfully being received after passing through the STAR Fire units using the oversampling technique. The 'Sink Statistics Graph' display shows the data rate and link utilisation against time. This shows a disconnection event when the SpaceWire cable used for the SpaceFibre links was removed and replaced. During this event, around time -10 seconds, no errors were detected by the SpaceWire validation software (as can be seen in the 'Sink statistics' window), showing that the link disconnection was handled as expected by SpaceFibre's FDIR capabilities and all errors that occur are recovered transparently.
WP5: SPACEWIRE-RT ASIC feasibility and virtual prototyping
From the SPACEWIRE-RT Outline Specification (WP2) and VHDL IP core design (WP4) the key issues for implementation of SPACEWIRE-RT in space qualifiable ASIC technology were identified:
1. size of memory required for virtual channel buffering which results in increasing ASIC size and power consumption, unless memory blocks are used;
2. data rate that can be handled by the physical layer drivers (CML or CML compatible and LVDS) and is limited by the space qualifiable ASIC technologies;
3. power consumption is less than 200 mW (175 mW for the 180nm SPACEWIRE-RT ASIC IP core) at increasing data rates in work range.
Available space qualifiable (radiation tolerant) ASIC technologies were reviewed and the area and timing parameters estimations for the ASIC technologies from 250 nm up to 65 nm were derived. The main results of the ASIC feasibility analysis are as follows:
Area:
The area (6.87 - 8.73 mm2) of the SPACEWIRE-RT ASIC IP-core on the base 180nm Radiation Tolerant Libraries is approximately 3-4 times larger than a SpaceWire ASIC IP-core (1-8 virtual channels). Of course, the SPACEWIRE-RT IP core includes all advantages of QoS, FDIR and other SpW-RT benefits as well as operating at a much higher data-rate than SpaceWire. The area for the memory for eight virtual channels is about 1.78 mm2 for the 180 nm technology and is expected to be reduced with the new Memory compiler for the Elvees RT Library.
Data rate:
The data-rate of the SPACEWIRE-RT ASIC IP-core that can be achieved on the base 180 nm Radiation Tolerant Libraries is from the lower level (e.g. 5 Mbp/s) up to 1.25 Gbits/s. Up to four lanes could be supported with this technology increasing the data rate to 5G bits/s with four lanes. A data rate of 2.5 Gbits/s is expected to be achievable on a 130 nm CMOS process with 10 Gbps achievable using four lanes.
The implementation of SPACEWIRE-RT IP core in space qualifiable ASIC technology has been shown to be feasible with data rates of 2.5 Gbit/s per lane being expected using 130 nm technology.
The analysis of WP5 results shows that the SPACEWIRE-RT RT ASIC IP core is effectively realisable on the basis of existing Russian and European Microelectronics Technologies.
WP6 SPACEWIRE-RT standard draft
During the SPACEWIRE-RT project the SpaceFibre standard was extended with the quality and network layers being added. Much of the SpaceFibre standard was simulation in SDL and SystemC and the results of the simulation used to help improve all layers of the standard specification. SpaceFibre was implemented as an IP core suitable for implementation and testing in an FPGA. The results of this hardware implementation were very useful in finding problems with and improving the SpaceFibre specification.
The SpaceFibre standard specification was updated to draft F and has been translated into Russian. It is expected that only relatively minor changes will be made to the quality and lane layers that form the core of the SpaceFibre specification. The network layer provides a baseline for further development and discussion of suitable network layer approaches.
SPACEWIRE-RT objectives addressed
The SPACEWIRE-RT objectives have been fully addressed:
1. a comprehensive set of requirements and use cases for spacecraft onboard networking has been produced;
2. the SpaceFibre networking technology has been selected as the basis for SPACEWIRE-RT and essential, innovative QoS and FDIR capabilities designed to complement SpaceFibre;
3. a baseline network layer for SpaceFibre has been devised;
4. a validated SPACEWIRE-RT specification has been written with important, novel features of SPACEWIRE-RT networks tested using SDL and SystemC models;
5. the feasibility of implementation in space qualifiable ASIC technologies has been assessed and demonstrated through simulation;
6. the results of the SPACEWIRE-RT study have been disseminated to the European and Russian space industries and to the international space community;
7. the SPACEWIRE-RT technology has been reviewed at various points during the two years of the SPACEWIRE-RT project by the International SpaceWire working group;
8. a draft standard document for SPACEWIRE-RT has been produced.
Potential impact:
SpaceWire is a European technology used by the world's space agencies and space industry on many spacecraft. The main objective of the SPACEWIRE-RT project was to take this technology to the next level, by providing an enhanced SpaceWire network technology that provides quality of service capabilities suitable for spacecraft data-handling and control applications, enabling it to support rapid spacecraft assembly and also making it applicable to other applications. The creation of this technology has substantially strengthen collaborative bonds between the Russian and European organisations involved in the research and led to technology of vital importance for future space missions.
Building on the SpaceFibre technology being designed by University of Dundee for ESA, the SPACEWIRE-RT project has developed a high-performance network technology that fulfils the needs of many spacecraft on-board communication requirements. Specifically the SPACEWIRE-RT project designed QoS and FDIR techniques for SpaceFibre and specified a baseline network layer. This research resulted in the specification of the quality and network layers for SpaceFibre. In addition the entire SpaceFibre protocol stack has been simulated and the results of this simulation used to improve the existing SpaceFibre protocols. A VHDL implementation of the quality layer has been developed and used to test this layer together with the rest of the SpaceFibre protocol stack, again resulting in improvements to the specification. The feasibility of implementation as flight qualified chips has been explored and shown to be achievable. The resulting inputs to the SpaceFibre specification have been presented to the international SpaceWire working group and incorporated into the draft SpaceFibre standard. It is expected that this will become a formal ECSS specification in 2014 - 2015.
The research on the quality layer has been very important for SpaceFibre, providing a network technology that is robust against failure, while retaining high-performance and which can support different types of traffic flowing over a single link. The QoS technique developed combines priority, bandwidth-reservation and scheduling techniques to support just about all spacecraft communication applications with a single network technology. Research on over-sampling has demonstrated that a SpaceFibre interface operating at 100 Mbits/s can be supported without requiring analogue phase-locked loop clock / data recovery technology, allowing it to be implemented in any FPGA technology.
SpaceFibre is likely to be adopted widely with implementations by European, Japanese and Russian organisations already underway. It will support very high-data rate missions like synthetic aperture radar and multi-spectral imaging. It will support integrated avionics applications with control and housekeeping information flowing over the same network as payload data, reducing overall power and mass budgets for the on-board communication network. SpaceFibre is backwards compatible with the packet level of SpaceWire and will support mixed SpaceWire / SpaceFibre systems allowing ready migration to the improved SpaceFibre network technology.
SpaceFibre is designed primarily for spacecraft applications, but its key characteristics of high-performance, low-latency, integrated QoS, integrated FDIR and implementation simplicity are likely to make it attractive for terrestrial applications too. For example, robotics applications will benefit significantly from SpaceFibre's capabilities.
SpaceFibre will underpin many future science, exploration, Earth observation and commercial missions. These missions relying on SpaceFibre technology will support new scientific discoveries, will help monitor and protect our environment and will enhance our quality of life through improved global telecommunications, positioning, disaster monitoring and weather prediction satellites.
The SPACEWIRE-RT project will have a positive economic impact on the small and medium sized enterprises (SME) and industrial partners who have been involved in the project. The partners are already planning to commercially exploit the results of the SPACEWIRE-RT project through new or improved products which will be available within the next one to seven years. The SME partners also expect this will have a positive impact on employment resulting in approximately 10 new jobs.
Finally, the SPACEWIRE-RT project will also have an educational impact with the academic partners planning to incorporate elements of the project results into their undergraduate and postgraduate courses.
Main dissemination activities
Dissemination took the following forms:
1. the SPACEWIRE-RT project website;
2. participation in SpaceWire working group meetings and scientific conferences;
3. project newsletters;
4. publications.
The SPACEWIRE-RT website (see http://spacewire-rt.org for details) provides information on the project for both the general public and the research community. The 'news' page is aimed at a wide audience and contains links to the project newsletters and news articles related to the project partners. The 'project and publications' pages are aimed more towards the research community. These pages contain an overview of the project objectives, downloadable copies of the non-confidential project deliverable reports and details of project-related papers presented at scientific conferences and SpaceWire working group meetings.
Over the course of the project, all partners have been involved in presenting the key features and concepts of SPACEWIRE-RT at a wide variety of forums. This included four SpaceWire working group meetings, regular meetings of the Russian National SpaceWire working group, six scientific conferences in Europe and Russia and three educational conferences in Russia. The working group meetings provided the main vehicle for gaining feedback and validating the future plans at each stage of the project and enabling early engagement with the end user community.
Dissemination activities have continued beyond the end of the project with a number of project-related papers being published in the Radioelectronics, Engineering Technology 2013 journal published in Russian by Elvees in June 2013 and a SpaceFibre tutorial is planned for the SpaceWire 2013 conference being held in June 2013.
Exploitation of results
The results of the SPACEWIRE-RT project will be exploited by the SPACEWIRE-RT partners in a number of ways:
SpaceFibre standard specification
The primary results of the SPACEWIRE-RT project, including the QoS and FDIR technology designed for SpaceFibre, has been included in the SpaceFibre standard specification. The University of Dundee will take this specification through the ESA ECSS standardisation process starting in 2014, to become a formal ECSS standard. SpaceFibre, incorporating the results of the SPACEWIRE-RT activity, is expected to become an important technology for future spacecraft on-board data handling applications.
Development of new or enhanced products based on the SPACEWIRE-RT technology
The SME and industrial partners plan to develop new products and enhance existing products based on the SPACEWIRE-RT technology.
STAR-Dundee Ltd, a successful spin-out company from the UnivDun, will commercialise the SPACEWIRE-RT technology, including the SpaceFibre VHDL IP core. It is expected that several chip designs incorporating the SpaceFibre VHDL IP core will be developed and used in many future space missions including multi-spectral imagers and synthetic aperture radar.
STAR-Dundee Ltd will also develop test equipment for SpaceFibre including the QoS and FDIR capabilities from SPACEWIRE-RT. The STAR-Dundee 'STAR fire' unit already provides interface and analysis capabilities for the current draft of SpaceFibre. Additional test equipment will be designed to support SpaceFibre adoption by the international aerospace industry.
Elvees, in collaboration with the industrial partners and Universities, plan to design a Radiation Tolerant SPACEWIRE-RT IP-library, SPACEWIRE-RT ASIC IP-cores and SPACEWIRE-RT based chipsets (Multicore DSP microprocessor with the integrated on the silicon SPACEWIRE-RT links router, microcontroller for Mass Storage with the integrated on the silicon SPACEWIRE-RT links router, multiports SPACEWIRE-RT router and multi-channel adapter) for a variety of microelectronics processes of the Russian-European factories.
Submicron will integrate the results of SPACEWIRE-RT into the design of onboard data systems and control equipment for spacecraft and failure-tolerant structure of onboard computing systems of prospective satellites and piloted transport spacecraft with high-speed information interfaces.
Astrium GmBH will design boards, boxes and systems containing SPACEWIRE-RT and SpaceWire networks for space applications.
Education and training
SpaceFibre will be used as an example of a high-performance network technology within the 'research frontiers' final year undergraduate computing course at the University of Dundee. Postgraduates and Internees within the University of Dundee Space Technology Centre and aerospace industry engineering staff will also receive training on SpaceFibre.
SUAI will make use of the SPACEWIRE-RT on-board networking technology in its education dedicated to aerospace technologies, space data systems and instruments engineering. SUAI will also provide training and maintenance during the uptake of the SPACEWIRE-RT technology by Russian space industry engineering staff.
Title: SPACEWIRE-RT
Coordinator:
Prof. Steve Parkes
Space Technology Centre, UnivDun
United Kingdom
Consortium:
SUAI, Russian Federation
http://www.suai.ru
SubMicron, Russian Federation
http://www.submicron.ru
Electronic VLSI Engineering and Embedded Systems, Russian Federation
http://multicore.ru/
Astrium GmbH, Germany
http://www.astrium.eads.net
Duration: 1 June 2011 - 31 May 2013 (24 months)
Funding Scheme: Seventh Framework Programme (FP7) Space, topic SPA.2010.3.2-04 'EU-Russia Cooperation for Strengthening Space Foundations'
Budget: European Union (EU) contribution: EUR 499 997
Website: http://spacewire-rt.org
For more information: spacewire@dundee.ac.uk
References
1. European Cooperation for Space Standardisation, Standard ECSS-E-50-12A, 'SpaceWire, Links, Nodes, Routers and networks', Issue 1, European Cooperation for Space Data Standardisation, January 2003
2. European Space Agency, 'SpaceWire Web Page', European Space Agency, http://spacewire.esa.int/
3. S. Parkes, 'SpaceWire User's Guide', available from http://www.star-dundee.com
4. S. Parkes, C. McClements, M. Dunstan and M. Suess, 'SpaceFibre: Gbits/s Link For Use On Board Spacecraft', International Astronautical Congress, Daejeon, South Korea, October 2009
5. S. Parkes and M. Suess, 'Virtual Channels, Broadcast Channels and SpaceFibre', International SpaceWire Conference, November 2011, San Antonio, USA, available from http://2011.spacewire-conference.org/proceedings/
6. S.Parkes A. Ferrer, A. Gonzalez-Villafranca, C. McClements, 'SpaceFibre Standard', Draft D, University of Dundee, February 2012, available from http://spacewire.esa.int/WG/SpaceWire/SpW-WG-Mtg18-Proceedings/
7. S.Parkes A. Ferrer, A. Gonzalez-Villafranca, C. McClements, 'SpaceFibre Standard', Draft E1, University of Dundee, September 2012, available from http://spacewire.esa.int/WG/SpaceWire/SpW-WG-Mtg19-Proceedings/
8. N. Sawyer, 'Data Recovery', Xilinx application note XAPP224 (v2.5) July 2005, available from http://www.xilinx.com
The requirements for spacecraft on-board communications were researched by the industrial partners covering both Russian and European requirements and a comprehensive set of use cases were defined. The SpaceFibre network technology being developed by the European Space Agency was identified as a good baseline for SPACEWIRE-RT. Enhancements to SpaceFibre necessary to meet the SPACEWIRE-RT requirements were identified and research undertaken in the areas of quality of service (QoS), fault detection, isolation and recovery (FDIR) and network layer concepts. The design for a simple, powerful and comprehensive quality of service mechanism resulted. This QoS mechanism was then extended to include specific classes of fault detection, 'babbling idiot' and quiet unit detection, in support of FDIR. Specifications for the SpaceFibre quality layer, covering QoS and FDIR and the network layer were written.
The entire SpaceFibre protocol stack was validated through simulated using specification and description language (SDL) and SystemC. The results of these detailed simulations were used to help remove errors and inconsistencies in the specification and to make necessary clarifications and improvements to the SpaceFibre standard. Several problems with the SpaceFibre specification were identified and potential solutions were explored and traded-off. The specification was updated accordingly.
A very-high-speed integrated circuits (VHSIC) hardware description language (VHDL) implementation of the SpaceFibre interface was designed and implemented in a field programmable gate array (FPGA). This provided a means of further testing and validating the SpaceFibre specification including its QoS and FDIR features. Results of this implementation fed back into the specification and further inconsistencies and problems were resolved, with the experimental FPGA implementation being updated and re-tested. The feasibility of ASIC implementation of the SpaceFibre was explored and a complete SpaceFibre interface was shown to be only three or four times the chip area of a SpaceWire interface.
The SpaceFibre specification was updated several times throughout the SPACEWIRE-RT project and a Russian translation made. It is planned for this specification to be taken through formal standardisation with the European Cooperation for Space Standardisation (ECSS) once other aspects of SpaceFibre have been completed, including multi-lane operation. Research and development related to SPACEWIRE-RT is being carried forward by the project partners with the aim of spaceflight ready components and system within the next two to three years.
Project context and objectives:
Context
The need for standard network technology for spacecraft applications
The trend towards 'Operationally responsive space', where spacecraft can be rapidly assembled, configured and deployed, to meet specific mission needs, e.g. disaster support, requires flexible, high-performance, on board communication networks with plug-and-play capability. Earth observation missions employing synthetic aperture radar or hyperspectral imaging have very high data rates requiring multi-gigabit/s onboard network technology. The growing autonomy of scientific missions to remote planets requires networks that are robust and durable, able to recover from transitory errors and faults automatically. The importance of spacecraft mass reduction motivates the sharing of networks for payload data-handling and avionics. Avionics and robotics applications impose requirements on network responsiveness and determinism. Increasing international collaboration on scientific and Earth observation spacecraft requires standard network technology where a component developed by one nation will interoperate effectively with equipment developed by another. SPACEWIRE-RT fulfils these demanding requirements with a flexible, robust, responsive, deterministic and durable standard network technology that is able to support both avionics and payload data-handling applications.
SpaceWire
The existing SpaceWire technology [1],[2],[3] was a very successful first step in this direction, providing networking technology for payload data-handling on over 40 major space missions. It falls short, however, of the deterministic and galvanic isolation requirements for avionics systems. SpaceWire is also of insufficient data-rate for some space applications.
SpaceFibre
SpaceFibre [4],[5],[6],[7] is a very high-speed serial data-link currently being developed by ESA which is intended for use in data-handling networks for high data-rate payloads. SpaceFibre is able to operate over fibre optic and electrical cable and support data rates of 2 Gbit/s in the near future and up to 20 Gbit/s long-term. It aims to complement the capabilities of the widely used SpaceWire onboard networking standard: improving the data rate by a factor of 10, reducing the cable mass by a factor of four and providing galvanic isolation. SpaceFibre aims to support QoS along with FDIR. An important feature of SpaceFibre is that it has the same packet format as SpaceWire, so that several SpaceWire links can be easily multiplexed over a single SpaceFibre link and application software designed for SpaceWire can operate over a SpaceFibre network. SpaceFibre is currently missing the important QoS and FDIR capabilities and the network layer of SpaceFibre has yet to be defined.
Project objectives
The SPACEWIRE-RT research programme aims to conceive and create communications network technology, suitable for a wide range of demanding space applications where responsiveness, determinism, robustness and durability are fundamental requirements. This is a critical component technology for future spacecraft avionics and payload data-handling. The SPACEWIRE-RT project substantially strengthened collaborative bonds between the Russian and European organisations involved in the research and led to technology of vital importance for future space missions.
The SPACEWIRE-RT research programme had the following key objectives:
1. a justified set of requirements and use cases for SPACEWIRE-RT covering both spacecraft payload and avionics, which takes into account requirements from European and Russian spacecraft primes and equipment manufacturers;
2. a conceptual design and outline specification for SPACEWIRE-RT that fulfils the requirements and use cases;
3. validation of the SPACEWIRE-RT specification, with important, novel features of SPACEWIRE-RT networks tested using SDL and System-C models;
4. a SPACEWIRE-RT VHDL intellectual property (IP) core tested and validated in an FPGA implementation;
5. an assessment of the feasibility of implementing SPACEWIRE-RT as an ASIC core considering available ASIC technologies suitable for space;
6. a draft standard document for SPACEWIRE-RT in both English and Russian;
7. dissemination of the results of the SPACEWIRE-RT study to the European and Russian space industries and to the international space community;
8. an exploitation plan covering the availability of flight grade chips, test and development equipment, IP cores and a programme for a test flight.
Project team
A highly experience team of Russian and European academic and industrial organisations was assembled for the SPACEWIRE-RT project. The project team is introduced below:
University of Dundee (UnivDun) has long experience in spacecraft onboard network technology, writing the SpaceWire standard with support from the European Space Agency (ESA) and input from engineers across Europe. UnivDun has extensive experience in related IP core design having designed SpaceWire interface, remote memory access protocol (RMAP) and router cores for ESA, JAXA and other organisations. The principal responsibilities of UnivDun were overall project management, conceptual design and specification, VHDL IP core design and SPACEWIRE-RT standard specification.
Astrium GmbH is a major international spacecraft manufacturer that undertakes system integration and also designs and manufactures spacecraft payloads. Astrium was responsible with Submicron for the requirements and use cases for SPACEWIRE-RT based on their extensive spacecraft design experience.
Submicron is a Russian organisation that specialises in the design and production of spacecraft avionics. Submicron was responsible with Astrium for the requirements and use cases for SPACEWIRE-RT based on their extensive experience of spaceflight electronic equipment.
St Petersburg University of Aerospace Instrumentation (SUAI) are experienced in the simulation and testing of network technologies using SDL particularly in the field of aerospace. SUAI designed their own SpaceWire IP cores and devised an innovative interrupt mechanism for SpaceWire. SUAI was primarily responsible for the simulation and validation of the SPACEWIRE-RT protocols and for translating the SPACEWIRE-RT standard specification in to Russian. SUAI also contributed to the evolution of the SPACEWIRE-RT specification.
Elvees is a leading Russian ASIC design house with expertise in space grade electronics and chip design. Elvees was responsible for reviewing the SPACEWIRE-RT specification and considering the feasibility of ASIC implementation.
Project Results:
Project overview
The technical work began with work package one (WP1) and WP2. WP1 (Spacecraft avionics and payload use cases) focussed on the requirements and use cases for spacecraft onboard communication networks for both payload applications and avionics. Requirements for avionics networks were gathered, covering reliability, fault tolerance, fault isolation, performance, responsiveness and determinism. A series of use cases was explored covering a diverse set of avionics and payload applications. These requirements formed the basis for the technical capabilities of the SPACEWIRE-RT network technology.
WP2 (Concept and specification) reviewed existing networking technology, including commercial technologies, SpaceWire, SpaceFibre and other onboard technologies. From the requirements of WP1, a set of evaluation criteria were derived and the candidate networking concepts traded-off against one another. SpaceFibre was selected as the baseline technology for SPACEWIRE-RT. Several areas of SpaceFibre which required substantial further research and development were identified for investigation in the SPACEWIRE-RT project, including QoS, FDIR and the network layer. Concepts were devised that met the requirements from WP1 and a specification produced.
In WP3 (Simulation and validation), simulation models covering key aspects of SPACEWIRE-RT were designed and used to evaluate the proposed SPACEWIRE-RT solution. The simulation models were developed in SDL and used to run various validation scenarios. The results of the simulation were used to update the SPACEWIRE-RT specification and to inform the VHDL IP core development (WP4) and ASIC feasibility (WP5) activities.
WP4 (VHDL IP core prototype) began with the architectural design of the SPACEWIRE-RT IP core based on information from the outline SPACEWIRE-RT specification (WP2). Once the architectural design was ready the design and development of the VHDL IP core and associated test bench was carried out, followed by testing. The IP core was implemented in an FPGA and tested extensively. The results were presented to the SpaceWire working group and the SPACEWIRE-RT specification updated.
WP5 (ASIC feasibility) covered the review and evaluation of suitable ASIC technologies for SPACEWIRE-RT implementation, the procurement of suitable ASIC libraries and initial design and simulation work. Owing to the expense of ASICs, device manufacture was beyond the scope of the present activity, however, the principal risk areas with an ASIC development were investigated and the feasibility of ASIC implementation assessed.
The results from all the previous WPs were fed into WP6 (Standard draft) where a draft standard for SPACEWIRE-RT was written by UnivDun with all partners involved in review of the document. The SPACEWIRE-RT standard built on the SpaceFibre specification, adding in the quality and network layers. The draft standard was presented to the SpaceWire working group and feedback used to improve the document. SUAI translated the standard into Russian.
WP1: Spacecraft avionics and payload use cases
Astrium GmbH and Submicron gathered requirements from spacecraft manufacturers and spacecraft equipment suppliers across Europe and Russia respectively. These requirements covered many aspects of payload data-handling and avionics networks.
An important aspect of this work on requirements was the need for a range of QoS depending on the particular application. These include the capability to reserve network bandwidth to prevent different data flows interfering with one another, deterministic data delivery for control applications and high priority, very low-latency communication for time-synchronisation and side-band signalling.
The requirements were analysed by University of Dundee and compared against the characteristics of the planned SpaceFibre standard. It was clear that SpaceFibre would meet the majority of the requirements for SPACEWIRE-RT and so was adopted as the baseline network technology for very high data-rate applications. The outcome of this analysis was used to inform the work on QoS within SpaceFibre and resulted in a coherent precedence concept being proposed for QoS in SpaceFibre.
WP2: Concepts and specification development
SPACEWIRE-RT and SpaceFibre
The SPACEWIRE-RT project built on the emerging SpaceFibre standard which is being designed by the University of Dundee for ESA with inputs from international spacecraft engineers. The aim is to publish the final standard though the ECSS. The SPACEWIRE-RT project has contributed to the following parts of the SpaceFibre standard:
1. QoS mechanisms in the quality layer;
2. FDIR mechanisms in the quality layer;
3. network level concepts in the network layer.
The QoS and FDIR work has been fully integrated in the SpaceFibre standard specification. A baseline network layer specification has been added to the SpaceFibre standard specification. The overall SpaceFibre protocol stack has been rationalised and the SpaceFibre standard restructured accordingly.
Several version of the SpaceFibre specification were written as the SPACEWIRE-RT project progressed, incorporating QoS, FDIR and network layer specifications. Extensive simulation of each of these specification versions was carried out in the SPACEWIRE-RT project to help improve and validate the SpaceFibre draft standard. Draft E1 of the standard [7] was used for the critical simulation of the Virtual Channel and Retry Layers. Draft D [6] was used for the lower layers because this simulation work was carried out before draft E1 of the SpaceFibre standard was written. The results of the simulations were used to improve the SpaceFibre specification.
SpaceFibre protocol stack
The network layer is responsible for the transfer of application information over a SpaceFibre network. It provides two services: packet transfer service and broadcast message service. The packet transfer service transfers SpaceFibre packets over the SpaceFibre network, using the same packet format and routing concepts as SpaceWire uses. SpaceFibre supports both path and logical addressing. The broadcast message service is responsible for broadcasting short messages (8 bytes) to all nodes on the network. These messages can carry time and synchronisation signals and be used to signal the occurrence of various events on the network.
The management layer is responsible for configuring, controlling and monitoring the status of all the layers in the SpaceFibre protocol stack. For example, it can configure the QoS settings of the virtual channels in the QoS and FDIR layer.
The quality layer is responsible for providing quality of service and managing the flow of information over a SpaceFibre link. It frames the information to be sent over the link to support QoS and scrambles the packet data to reduce electromagnetic emissions. The QoS and FDIR layer also provides a retry capability, detecting any frames or control codes that go missing or arrive containing errors and resending them. With this inbuilt retry mechanism SpaceFibre is very resilient to transient errors.
The multi-lane layer is responsible for operating several SpaceFibre lanes in parallel to provide higher data throughput. In the event of a lane failing the multi-lane layer provides support for graceful degradation, automatically spreading the traffic over the remaining working links.
The lane layer is responsible for lane initialisation and error detection. In the event of an error the lane is automatically re-initialised. The lane layer encodes data into symbols for transmission using 8B/10B encoding and decodes these symbols in the receiver. 8B/10B codes are direct current (DC) balanced supporting alternating current (AC) coupling of SpaceFibre interfaces.
The physical layer is responsible for serialising the 8B / 10B symbols and for sending them over the physical medium. In the receiver the Physical layer recovers the clock and data from the serial bit stream, determines the symbol boundaries and recovers the 8B / 10B symbols. Both electrical cables and fibre-optic cables are supported by SpaceFibre.
The specific results of the SPACEWIRE-RT project that contributed to the SpaceFibre standard specification will now be described.
SpaceFibre quality of service
The specification, validation, prototype implementation and testing of the SpaceFibre quality layer was a major output of the SPACEWIRE-RT project. In this section, a detailed overview of this technology is provided.
Frames and virtual channels
To provide quality of service, it is necessary to be able to interleave different data flows over a data link or network. If a large packet is being sent with low priority and a higher priority one requests to be sent, it must be possible to suspend sending the low priority one and start sending the higher priority packet. To facilitate this SpaceWire packets are chopped up into smaller data units called frames. When the high priority packet requests to be sent, the current frame of the low priority packet is allowed to complete transmission and then the frames of the high priority packet are sent. When all the frames of the high priority packet have been sent, the remaining frames of the low priority packet can be sent.
Each frame has to be identified as belonging to a particular data flow so that the stream of packets can be reconstructed at the other end of the link. Low priority packets belong to one data stream and high priority packets belong to another data stream.
Each independent data stream allowed to flow over a data link is referred to as a virtual channel (VC). Virtual channels are unidirectional and have a QoS attribute, e.g. priority. At each end of a virtual channel is a virtual channel buffer (VCB), which buffers the data from and to the application. An output VCB takes data from the application and buffers it prior to sending it across the data link. An input VCB receives data from the data link and buffers it prior to passing it to the receiving application.
There can be several output virtual channels connected to a single data link, which compete for sending information over the link. A medium access controller determines which output virtual channel is allowed to send the next data frame. When an output VCB has a frame of data ready to send and the corresponding input VCB at the other end of the link has room for a full data frame, the output VCB requests the medium access controller to send a frame. The medium access controller arbitrates between all the output VCBs requesting to send a frame. It uses the QoS attribute of each of the requesting VCBs to determine which one will be allowed to send the next data frame.
Priority is one example of a QoS attribute. Other types of QoS are considered in the subsequent sections.
Precedence
For the medium access controller to be able to compare QoS attributes from different output VCBs, it is essential that they are all using a common measure that can be compared. The name given to this measure is precedence. The competing output VCB with the highest precedence will be allowed to send the next frame.
bandwidth reservation
When connecting an instrument via a network to a mass memory, what the systems engineer needs to know is 'how much bandwidth do I have to transfer data from the instrument to the mass memory?' Once the network bandwidth allocated to a particular instrument has been specified, it should not be possible for another instrument to impose on the bandwidth allocated to that instrument. A priority mechanism is not suitable for this application. If an instrument with high priority has data to send, it will hog the network until all its data has been sent. What is needed is a mechanism that allows bandwidth to be reserved for a particular instrument.
Bandwidth reservation calculates the bandwidth used by a particular virtual channel and compares this to the bandwidth reserved for that virtual channel to calculate the precedence for that virtual channel. If the virtual channel has not used much reserved bandwidth recently, it will have a high precedence. When a data frame is sent by this virtual channel, its precedence will drop. Its precedence will increase again over a period of time. If a virtual channel has used more than its reserved bandwidth recently, it will have a low precedence.
A virtual channel specifies a portion of overall link bandwidth that it wishes to reserve and expects to use, i.e. its expected bandwidth.
When a frame of data is send by any virtual channel, each virtual channel computes the amount of bandwidth that it would have been permitted to send in the time interval that the last frame was sent. This is known as the bandwidth allocation.
Each virtual channel can use this to determine its bandwidth credit, which is effectively the amount of data it can send and still remain within its expected bandwidth. Bandwidth credit is the bandwidth allowance less the bandwidth used accumulated over time.
The bandwidth credit is updated every time a data frame for any virtual channel has been sent. A bandwidth credit value close to zero indicates nominal use of bandwidth by the virtual channel. A negative value indicates that the virtual channel is using more than its expected amount of link bandwidth. A positive value indicates that the virtual channel is using less than its expected amount of link bandwidth.
To simplify the hardware required to calculate the bandwidth credit it is allowed to saturate at plus or minus a bandwidth credit limit, i.e. if the bandwidth credit reaches a bandwidth credit limit it is set to the value of the bandwidth credit limit.
When the bandwidth credit for a virtual channel reaches the negative bandwidth credit limit it indicates that the virtual channel is using more bandwidth than expected. This may be recorded in a status register and used to indicate a possible error condition. A network management application is able to use this information to check correct utilisation of link bandwidth by its various virtual channels.
For a virtual channel supporting bandwidth reserved QoS, the value of the bandwidth counter provides the precedence value for that virtual channel.
If the bandwidth credit counter reaches the minimum possible bandwidth credit value, it indicates that it is using more bandwidth than expected and a possible error may be flagged. This condition may be used to stop the VC sending any more data until it recovers some bandwidth credit, to help with 'babbling idiot' protection.
Similarly if the bandwidth credit counter stays at the maximum possible bandwidth credit value for a relatively long period of time, the VC is using less bandwidth than expected and this condition can be flagged to indicate a possible error.
The bandwidth credit value is the precedence used by the medium access controller to determine which VC is permitted to send the next data frame.
Priority
The second type of QoS provided by VCs is priority. Each VC is assigned a priority value and the VC with the highest priority (lowest priority number) is allowed to send the next data frame as soon as it is ready.
Within any level there can be any number of VCs which compete amongst themselves based on their bandwidth credit. A higher priority VC will always have precedence over a lower priority VC unless its bandwidth credit has reached the minimum credit limit in which case it is no longer allowed to send any more data frames. This prevents a high priority VC from consuming all the link bandwidth if it fails and starts babbling. More than one VC can be set to the same priority level in which case those VCs will compete for medium access using bandwidth reservation.
Scheduled
To provide fully deterministic data delivery it is necessary for the QoS mechanism to ensure that data from specific virtual channels can be sent (and delivered) at particular times. This can be done by chopping time into a series of time-slots, during which a particular VC is permitted to send data frames.
Each VC is allocated one or more time-slots in which it is permitted to send data frames. VC1 is scheduled to send in time-slot one and VC2 is scheduled to send in time-slots two and six. The time-slot duration is a system level parameter, typically 100 µs and there are 256 time-slots.
During a time-slot, if the VC is scheduled to send in that time-slot, it will compete with other VCs also scheduled to send in that time-slot based on precedence (priority and bandwidth credit). A fully deterministic system would have one VC allowed to send in each time-slot.
The schedule is always operating. If a user does not want to use scheduling the schedule table is simply filled completely, allowing any VC to send in any time-slot, competing with precedence.
Scheduling can waste bandwidth if only one VC is allowed to send in a time-slot and that VC is not ready. To avoid this situation, the critical VC can be allocated a time-slot and given high priority and another VC can be allocated the same time-slot with lower priority. In this way when that time-slot arrives the high priority VC will be allowed to send its data, but if it is not ready the VC with lower priority can send some data.
Time-slots can be defined by broadcasting start of time-slot signals, or by broadcasting time and having a local time counter which determines the start and end of each time-slot. The SpaceFibre broadcast message mechanism supports both synchronisation and time distribution.
The SpaceFibre QoS mechanism is simple and efficient to implement and it provides bandwidth reservation, priority and scheduling integrated together, not as separate options. Furthermore SpaceFibre QoS provides a means for detecting 'babbling idiots' and for detecting nodes that have ceased sending data when they are expected to be sending information.
SpaceFibre FDIR
The FDIR mechanism for SpaceFibre is another significant output of the SPACEWIRE-RT project. A brief overview of the FDIR for SpaceFibre is provided in this section.
SpaceFibre provides automatic fault detection, isolation and recovery. When a fault occurs on a SpaceFibre link, it is detected and the erroneous or missing information resent. SpaceFibre recovers from intermittent faults very rapidly, detecting faults, recovering and resending data faster than SpaceWire disconnects and reconnects a link. The retry mechanism does not depend on time-outs, naturally adapting to different cable delays.
Fault detection is provided by checking each 8B / 10B symbol for disparity errors and invalid 8B / 10B codes. SpaceFibre has selected the 8B / 10B K-codes it uses to have enhanced Hamming distance from data-codes. This means that a single bit error occurring in a data-code cannot result in a valid K-code used by SpaceFibre. In addition each data frame, broadcast frame, FCT, ACK and NACK are protected by a CRC.
Fault isolation is provided at various levels in SpaceFibre. AC coupling is used in the physical layer to prevent damage from faults that cause DC voltages exceeding the maximum permitted to appear on the transmitter outputs or receiver inputs. This feature also enables galvanic isolation to be implemented readily. At the quality level SpaceFibre provides time containment, containing errors in the data frame in which they occur and bandwidth containment, containing errors to the virtual channel in which they occur; an error in one VC does not affect data flowing in another VC. Babbling idiots are contained using the QoS mechanism described above.
Fault recovery is provided at the link level using a retry mechanism that resends data frames, broadcast frames and FCTs. The retry is very fast, uses a minimum amount of buffer memory and adapts automatically to different link lengths. In addition to the retry mechanism the multi-lane functionality includes graceful degradation on lane failure. If a lane fails permanently, so that a retry or re-initialisation does not recover lane operation, a multi-lane system will continue using the remaining lanes available. This reduces the bandwidth available but does not stop the link operating. For critical operations an extra lane can be included and the graceful degradation will then provide automatic replacement of a faulty lane. The bit error rate (BER) of a lane is monitored and a lane reported as faulty if the BER is above a level which results in the effective link bandwidth being unusable. This feature allows lanes that can re-initialise successfully but which will not run for very long before having to re-initialise again, to be detected, isolated and replace by a fully functional lane.
SpaceFibre networks
The third substantial contribution of the SPACEWIRE-RT project is a baseline concept for the SpaceFibre network layer. Initial validation of the network layer has been carried out by SUAI using SystemC.
A SpaceFibre network uses similar packet formats, packet addressing and routing concepts as SpaceWire. The main difference is that SpaceFibre includes virtual channels.
The SpaceFibre router comprises a number of SpaceFibre interfaces and a routing switch matrix. Each SpaceFibre interface has several virtual channels. The VC number for each virtual channel can be configured, except for VC0 which is a virtual channel used for configuration, control and monitoring of the SpaceFibre network. When a packet arrives on a SpaceFibre interface it is placed in the appropriate virtual channel, i.e. the one with the same VC number as it was transmitted on. The leading data character of the packet determines which port of the routing switch the packet is to be forwarded through using either path or logical addressing. The port that it is to be switched to must have a VC configured with the same number as the VC that the packet arrived on. The packet is then passed through the routing switch matrix and placed frame by frame in the VC of the output port. The packet is then transferred across the SpaceFibre link, competing with other VCs in that port for access to the link medium according to their precedence.
If a packet arrives and the output port that the packet is to be switched to does not have a VC with the same number as that on which it arrived, the packet is spilt and an error recorded.
Virtual channels can be used to construct virtual networks where a single VC number is used for connecting to all or several of the nodes attached to the network. Virtual channels can also be used to construct virtual point to point links from one node to another.
Physical layer
SpaceFibre is a very high-speed communication protocol. The QoS and FDIR protocols developed for SpaceFibre can potentially be applied to a range of slower speed network technologies. If a consistent set of QoS and FDIR protocols can be applied across network technologies like SpaceWire then the concepts will be common and application software can be readily reused regardless of the performance required from the network. SPACEWIRE-RT aims to use the QoS and FDIR protocols being developed for SpaceFibre across a range of different lower level protocol layers to cover a range of data rates and applications.
At present two physical layer protocols are being considered:
1. SpaceFibre current mode logic (CML):
The SpaceFibre protocol is designed to run at multiple Gbits/s using CML over copper cables for distances up to 5 m or fibre optic cables for distances of 100 m or more. Multiple lanes can be operated in parallel to increase the data rate. SpaceFibre uses 8B / 10B encoding and a phase locked loop (PLL), or similar technology, to recover a clock from the received data stream and to perform bit synchronisation.
2. SpaceFibre LVDS:
The SpaceFibre protocol could also be run at slower speed using low voltage differential signalling (LVDS), which is capable of 600 Mbits/s or possibly up to 1 Gbits/s data rates. LVDS is used in SpaceWire and therefore has space heritage. SpaceFibre LVDS would use 8B/10B encoding and a PLL for bit synchronisation.
It is proposed that both the CML and LVDS options for SpaceFibre are included in the SpaceFibre standard specification.
Oversampled SpaceFibre
SpaceFibre requires clock-data recovery circuitry in the receiver to recover the clock from the received data bit-stream. Normally, this is accomplished using some form of phase-locked loop (PLL) which requires specialised circuitry. Current space qualified FPGAs do not provide suitable PLL clock recovery circuitry, so an alternative clock recovery scheme is required if SPACEWIRE-RT is to be implemented in space qualified FPGAs. The receive signal can be over sampled (sampled with a clock of significantly higher frequency than the receive bit rate) and bit synchronisation achieved [8]. Data rates of up to 100 Mbits/s should be achievable, which is adequate for many spacecraft applications. This oversampling technique would simply be an alternative implementation for the clock-data recovery for SpaceFibre and be suitable at relatively low data-rates (e.g. 100 Mbits/s or slower).
WP3: SPACEWIRE-RT validation and simulation
One of the main objectives of the WP3 was to check the correctness and consistency of the SPACEWIRE-RT specification. This was done by specification and simulation using SDL for the lower layers of SpaceFibre and by simulation using SystemC for the network layer. Simulation and investigation was split into two activities: checking correctness and consistency of SPACEWIRE-RT and testing against requirements. From the start of WP3, three drafts of the SpaceFibre specification were released which were taken as the basis for the SPACEWIRE-RT standard. During specification and simulation several documents with proposals, change requests and issues for discussion were produced and provided to the University of Dundee. Overall, SUAI produced 139 change requests, 87 of which resulted in changes to the SpaceFibre standard. In this way the results of specification and simulation in SDL and SystemC were applied during the SPACEWIRE-RT development.
In the scope of testing SPACEWIRE-RT against the requirements of industry, a number of tests were performed, which evaluated and checked SPACEWIRE-RT functionality in point-to-point connection (primarily SDL model) and in networks with different topologies (SystemC model). To test basic mechanisms of the SPACEWIRE-RT two different network configurations were used: point-to-point configuration and mixed configuration (tree and circular configurations combined in one network). Overall this testing showed that SPACEWIRE-RT satisfies the requirements provided by the industry.
SDL point-to-point model
The SDL model was primarily focussed on checking internal functionality and how all mechanisms of all layers work in common in one node. The SDL model was layered according to the SpaceFibre specification and service access points (SAPs) were specified as the interfaces between layers.
SystemC network model
SystemC network models were primarily focussed on simulation and checking of network aspects of the SPACEWIRE-RT, performance parameters (e.g. latency) and some features that cannot be covered by SDL point-to-point simulation (e.g. broadcast distribution, QoS, etc.).
WP4: SPACEWIRE-RT VHDL IP core development
SpaceFibre VHDL IP core
The QoS and FDIR capabilities for SpaceFibre were implemented in an experimental SpaceFibre VHDL IP core so that the concepts could be tested in hardware. The design was implemented in a STAR Fire unit from STAR-Dundee, which contained a suitable FPGA with SpaceWire and SpaceFibre connectors attached to the FPGA. The design was extensively tested using the inbuilt SpaceFibre link analysis capabilities of the STAR Fire unit.
Following testing of the hardware implementation, improvements were made to the SpaceFibre specification and the SpaceFibre VHDL IP core was updated and re-tested. The design, simulation, implementation, test and update to the specification cycle were iterated several times as problems were discovered, solutions devised and the specification polished. This cycle of improvement ran concurrently with the improvements resulting from the simulation of the standard specification in WP3. The black cables plugged into the front panel are the SpaceFibre electrical cables. The FPGA is the largest chip on the right hand side of the unit.
Oversample SpaceFibre VHDL IP core
An oversampling technique suitable for SPACEWIRE-RT based on [8] was designed, simulated, implemented and tested in an FPGA. SpaceFibre ran successfully in the FPGA using oversampling for clock-data recovery.
The design was implemented in a pair of STAR-Dundee STAR Fire units. The two STAR Fire units were modified so that each unit had one normal SpaceWire interface (Port 1) and one oversampled SpaceFibre interface (port 2). A host PC provided a stream of SpaceWire packets via a SpaceWire USB Brick which were sent to the SpaceWire interface on one STAR Fire unit. The SpaceWire packets were then sent out of the Oversampled SpaceFibre interface to the other STAR Fire unit. The SpaceWire packets were received and looped-back using a SpaceWire router inside the STAR Fire unit so that they were sent back out of the Oversampled SpaceFibre interface. The SpaceWire packets were received back at the first STAR Fire unit and passed up to the host PC for checking, via the SpaceWire USB Brick.
Software running on the host PC checked the SpaceWire packets received noting any errors. The 'Sink Statistics' dialog box displays the total data packets received, data rate statistics and any detected errors. This shows that packets are successfully being received after passing through the STAR Fire units using the oversampling technique. The 'Sink Statistics Graph' display shows the data rate and link utilisation against time. This shows a disconnection event when the SpaceWire cable used for the SpaceFibre links was removed and replaced. During this event, around time -10 seconds, no errors were detected by the SpaceWire validation software (as can be seen in the 'Sink statistics' window), showing that the link disconnection was handled as expected by SpaceFibre's FDIR capabilities and all errors that occur are recovered transparently.
WP5: SPACEWIRE-RT ASIC feasibility and virtual prototyping
From the SPACEWIRE-RT Outline Specification (WP2) and VHDL IP core design (WP4) the key issues for implementation of SPACEWIRE-RT in space qualifiable ASIC technology were identified:
1. size of memory required for virtual channel buffering which results in increasing ASIC size and power consumption, unless memory blocks are used;
2. data rate that can be handled by the physical layer drivers (CML or CML compatible and LVDS) and is limited by the space qualifiable ASIC technologies;
3. power consumption is less than 200 mW (175 mW for the 180nm SPACEWIRE-RT ASIC IP core) at increasing data rates in work range.
Available space qualifiable (radiation tolerant) ASIC technologies were reviewed and the area and timing parameters estimations for the ASIC technologies from 250 nm up to 65 nm were derived. The main results of the ASIC feasibility analysis are as follows:
Area:
The area (6.87 - 8.73 mm2) of the SPACEWIRE-RT ASIC IP-core on the base 180nm Radiation Tolerant Libraries is approximately 3-4 times larger than a SpaceWire ASIC IP-core (1-8 virtual channels). Of course, the SPACEWIRE-RT IP core includes all advantages of QoS, FDIR and other SpW-RT benefits as well as operating at a much higher data-rate than SpaceWire. The area for the memory for eight virtual channels is about 1.78 mm2 for the 180 nm technology and is expected to be reduced with the new Memory compiler for the Elvees RT Library.
Data rate:
The data-rate of the SPACEWIRE-RT ASIC IP-core that can be achieved on the base 180 nm Radiation Tolerant Libraries is from the lower level (e.g. 5 Mbp/s) up to 1.25 Gbits/s. Up to four lanes could be supported with this technology increasing the data rate to 5G bits/s with four lanes. A data rate of 2.5 Gbits/s is expected to be achievable on a 130 nm CMOS process with 10 Gbps achievable using four lanes.
The implementation of SPACEWIRE-RT IP core in space qualifiable ASIC technology has been shown to be feasible with data rates of 2.5 Gbit/s per lane being expected using 130 nm technology.
The analysis of WP5 results shows that the SPACEWIRE-RT RT ASIC IP core is effectively realisable on the basis of existing Russian and European Microelectronics Technologies.
WP6 SPACEWIRE-RT standard draft
During the SPACEWIRE-RT project the SpaceFibre standard was extended with the quality and network layers being added. Much of the SpaceFibre standard was simulation in SDL and SystemC and the results of the simulation used to help improve all layers of the standard specification. SpaceFibre was implemented as an IP core suitable for implementation and testing in an FPGA. The results of this hardware implementation were very useful in finding problems with and improving the SpaceFibre specification.
The SpaceFibre standard specification was updated to draft F and has been translated into Russian. It is expected that only relatively minor changes will be made to the quality and lane layers that form the core of the SpaceFibre specification. The network layer provides a baseline for further development and discussion of suitable network layer approaches.
SPACEWIRE-RT objectives addressed
The SPACEWIRE-RT objectives have been fully addressed:
1. a comprehensive set of requirements and use cases for spacecraft onboard networking has been produced;
2. the SpaceFibre networking technology has been selected as the basis for SPACEWIRE-RT and essential, innovative QoS and FDIR capabilities designed to complement SpaceFibre;
3. a baseline network layer for SpaceFibre has been devised;
4. a validated SPACEWIRE-RT specification has been written with important, novel features of SPACEWIRE-RT networks tested using SDL and SystemC models;
5. the feasibility of implementation in space qualifiable ASIC technologies has been assessed and demonstrated through simulation;
6. the results of the SPACEWIRE-RT study have been disseminated to the European and Russian space industries and to the international space community;
7. the SPACEWIRE-RT technology has been reviewed at various points during the two years of the SPACEWIRE-RT project by the International SpaceWire working group;
8. a draft standard document for SPACEWIRE-RT has been produced.
Potential impact:
SpaceWire is a European technology used by the world's space agencies and space industry on many spacecraft. The main objective of the SPACEWIRE-RT project was to take this technology to the next level, by providing an enhanced SpaceWire network technology that provides quality of service capabilities suitable for spacecraft data-handling and control applications, enabling it to support rapid spacecraft assembly and also making it applicable to other applications. The creation of this technology has substantially strengthen collaborative bonds between the Russian and European organisations involved in the research and led to technology of vital importance for future space missions.
Building on the SpaceFibre technology being designed by University of Dundee for ESA, the SPACEWIRE-RT project has developed a high-performance network technology that fulfils the needs of many spacecraft on-board communication requirements. Specifically the SPACEWIRE-RT project designed QoS and FDIR techniques for SpaceFibre and specified a baseline network layer. This research resulted in the specification of the quality and network layers for SpaceFibre. In addition the entire SpaceFibre protocol stack has been simulated and the results of this simulation used to improve the existing SpaceFibre protocols. A VHDL implementation of the quality layer has been developed and used to test this layer together with the rest of the SpaceFibre protocol stack, again resulting in improvements to the specification. The feasibility of implementation as flight qualified chips has been explored and shown to be achievable. The resulting inputs to the SpaceFibre specification have been presented to the international SpaceWire working group and incorporated into the draft SpaceFibre standard. It is expected that this will become a formal ECSS specification in 2014 - 2015.
The research on the quality layer has been very important for SpaceFibre, providing a network technology that is robust against failure, while retaining high-performance and which can support different types of traffic flowing over a single link. The QoS technique developed combines priority, bandwidth-reservation and scheduling techniques to support just about all spacecraft communication applications with a single network technology. Research on over-sampling has demonstrated that a SpaceFibre interface operating at 100 Mbits/s can be supported without requiring analogue phase-locked loop clock / data recovery technology, allowing it to be implemented in any FPGA technology.
SpaceFibre is likely to be adopted widely with implementations by European, Japanese and Russian organisations already underway. It will support very high-data rate missions like synthetic aperture radar and multi-spectral imaging. It will support integrated avionics applications with control and housekeeping information flowing over the same network as payload data, reducing overall power and mass budgets for the on-board communication network. SpaceFibre is backwards compatible with the packet level of SpaceWire and will support mixed SpaceWire / SpaceFibre systems allowing ready migration to the improved SpaceFibre network technology.
SpaceFibre is designed primarily for spacecraft applications, but its key characteristics of high-performance, low-latency, integrated QoS, integrated FDIR and implementation simplicity are likely to make it attractive for terrestrial applications too. For example, robotics applications will benefit significantly from SpaceFibre's capabilities.
SpaceFibre will underpin many future science, exploration, Earth observation and commercial missions. These missions relying on SpaceFibre technology will support new scientific discoveries, will help monitor and protect our environment and will enhance our quality of life through improved global telecommunications, positioning, disaster monitoring and weather prediction satellites.
The SPACEWIRE-RT project will have a positive economic impact on the small and medium sized enterprises (SME) and industrial partners who have been involved in the project. The partners are already planning to commercially exploit the results of the SPACEWIRE-RT project through new or improved products which will be available within the next one to seven years. The SME partners also expect this will have a positive impact on employment resulting in approximately 10 new jobs.
Finally, the SPACEWIRE-RT project will also have an educational impact with the academic partners planning to incorporate elements of the project results into their undergraduate and postgraduate courses.
Main dissemination activities
Dissemination took the following forms:
1. the SPACEWIRE-RT project website;
2. participation in SpaceWire working group meetings and scientific conferences;
3. project newsletters;
4. publications.
The SPACEWIRE-RT website (see http://spacewire-rt.org for details) provides information on the project for both the general public and the research community. The 'news' page is aimed at a wide audience and contains links to the project newsletters and news articles related to the project partners. The 'project and publications' pages are aimed more towards the research community. These pages contain an overview of the project objectives, downloadable copies of the non-confidential project deliverable reports and details of project-related papers presented at scientific conferences and SpaceWire working group meetings.
Over the course of the project, all partners have been involved in presenting the key features and concepts of SPACEWIRE-RT at a wide variety of forums. This included four SpaceWire working group meetings, regular meetings of the Russian National SpaceWire working group, six scientific conferences in Europe and Russia and three educational conferences in Russia. The working group meetings provided the main vehicle for gaining feedback and validating the future plans at each stage of the project and enabling early engagement with the end user community.
Dissemination activities have continued beyond the end of the project with a number of project-related papers being published in the Radioelectronics, Engineering Technology 2013 journal published in Russian by Elvees in June 2013 and a SpaceFibre tutorial is planned for the SpaceWire 2013 conference being held in June 2013.
Exploitation of results
The results of the SPACEWIRE-RT project will be exploited by the SPACEWIRE-RT partners in a number of ways:
SpaceFibre standard specification
The primary results of the SPACEWIRE-RT project, including the QoS and FDIR technology designed for SpaceFibre, has been included in the SpaceFibre standard specification. The University of Dundee will take this specification through the ESA ECSS standardisation process starting in 2014, to become a formal ECSS standard. SpaceFibre, incorporating the results of the SPACEWIRE-RT activity, is expected to become an important technology for future spacecraft on-board data handling applications.
Development of new or enhanced products based on the SPACEWIRE-RT technology
The SME and industrial partners plan to develop new products and enhance existing products based on the SPACEWIRE-RT technology.
STAR-Dundee Ltd, a successful spin-out company from the UnivDun, will commercialise the SPACEWIRE-RT technology, including the SpaceFibre VHDL IP core. It is expected that several chip designs incorporating the SpaceFibre VHDL IP core will be developed and used in many future space missions including multi-spectral imagers and synthetic aperture radar.
STAR-Dundee Ltd will also develop test equipment for SpaceFibre including the QoS and FDIR capabilities from SPACEWIRE-RT. The STAR-Dundee 'STAR fire' unit already provides interface and analysis capabilities for the current draft of SpaceFibre. Additional test equipment will be designed to support SpaceFibre adoption by the international aerospace industry.
Elvees, in collaboration with the industrial partners and Universities, plan to design a Radiation Tolerant SPACEWIRE-RT IP-library, SPACEWIRE-RT ASIC IP-cores and SPACEWIRE-RT based chipsets (Multicore DSP microprocessor with the integrated on the silicon SPACEWIRE-RT links router, microcontroller for Mass Storage with the integrated on the silicon SPACEWIRE-RT links router, multiports SPACEWIRE-RT router and multi-channel adapter) for a variety of microelectronics processes of the Russian-European factories.
Submicron will integrate the results of SPACEWIRE-RT into the design of onboard data systems and control equipment for spacecraft and failure-tolerant structure of onboard computing systems of prospective satellites and piloted transport spacecraft with high-speed information interfaces.
Astrium GmBH will design boards, boxes and systems containing SPACEWIRE-RT and SpaceWire networks for space applications.
Education and training
SpaceFibre will be used as an example of a high-performance network technology within the 'research frontiers' final year undergraduate computing course at the University of Dundee. Postgraduates and Internees within the University of Dundee Space Technology Centre and aerospace industry engineering staff will also receive training on SpaceFibre.
SUAI will make use of the SPACEWIRE-RT on-board networking technology in its education dedicated to aerospace technologies, space data systems and instruments engineering. SUAI will also provide training and maintenance during the uptake of the SPACEWIRE-RT technology by Russian space industry engineering staff.
Title: SPACEWIRE-RT
Coordinator:
Prof. Steve Parkes
Space Technology Centre, UnivDun
United Kingdom
Consortium:
SUAI, Russian Federation
http://www.suai.ru
SubMicron, Russian Federation
http://www.submicron.ru
Electronic VLSI Engineering and Embedded Systems, Russian Federation
http://multicore.ru/
Astrium GmbH, Germany
http://www.astrium.eads.net
Duration: 1 June 2011 - 31 May 2013 (24 months)
Funding Scheme: Seventh Framework Programme (FP7) Space, topic SPA.2010.3.2-04 'EU-Russia Cooperation for Strengthening Space Foundations'
Budget: European Union (EU) contribution: EUR 499 997
Website: http://spacewire-rt.org
For more information: spacewire@dundee.ac.uk
References
1. European Cooperation for Space Standardisation, Standard ECSS-E-50-12A, 'SpaceWire, Links, Nodes, Routers and networks', Issue 1, European Cooperation for Space Data Standardisation, January 2003
2. European Space Agency, 'SpaceWire Web Page', European Space Agency, http://spacewire.esa.int/
3. S. Parkes, 'SpaceWire User's Guide', available from http://www.star-dundee.com
4. S. Parkes, C. McClements, M. Dunstan and M. Suess, 'SpaceFibre: Gbits/s Link For Use On Board Spacecraft', International Astronautical Congress, Daejeon, South Korea, October 2009
5. S. Parkes and M. Suess, 'Virtual Channels, Broadcast Channels and SpaceFibre', International SpaceWire Conference, November 2011, San Antonio, USA, available from http://2011.spacewire-conference.org/proceedings/
6. S.Parkes A. Ferrer, A. Gonzalez-Villafranca, C. McClements, 'SpaceFibre Standard', Draft D, University of Dundee, February 2012, available from http://spacewire.esa.int/WG/SpaceWire/SpW-WG-Mtg18-Proceedings/
7. S.Parkes A. Ferrer, A. Gonzalez-Villafranca, C. McClements, 'SpaceFibre Standard', Draft E1, University of Dundee, September 2012, available from http://spacewire.esa.int/WG/SpaceWire/SpW-WG-Mtg19-Proceedings/
8. N. Sawyer, 'Data Recovery', Xilinx application note XAPP224 (v2.5) July 2005, available from http://www.xilinx.com
