Periodic Reporting for period 2 - NEPHELE (eNd to End scalable and dynamically reconfigurable oPtical arcHitecture for application-awarE SDN cLoud datacentErs)
Período documentado: 2016-08-01 hasta 2018-01-31
Datacenter network (DCN) traffic is skyrocketing, scaling rapidly at rates that reach 25% annually. Industry grapples to keep pace with this explosive growth that challenges the bandwidth and energy scalability of technologies currently used in DCNs. To avoid an imminent capacity crunch and improve power efficiency in DCNs new technologies are on the spotlight. Optical switching is gaining traction due to its inherent advantages; however, its practical deployment necessitates extensive modifications to the network architecture and operation.
European project NEPHELE is developing an optical network infrastructure that leverages optical switching within a software-defined networking (SDN) framework to overcome the bandwidth and energy scaling challenges of datacenter networks. NEPHELE relies on commercial off the shelf (COTS) photonic components for its data plane in order to expedite its deployment cycle while at the same time leverages open source control plane frameworks to maximize compatibility with existing infrastructures. NEPHELE introduces a dynamic optical network infrastructure for scale-out, disaggregated datacenters. NEPHELE builds on the enormous capacity of optical links and leverages hybrid optical-electronic switching to scale bandwidth at reduced cost and power compared to current technologies. In order to effectively integrate the new paradigm of optical switching into the datacenter networking ecosystem, NEPHELE followed an end-to-end approach extending from the datacenter architecture to the overlaying control plane and up to the interfaces with the application, in order to deliver a fully functional networking solution. The project brought together a multi-disciplinary workplan that introduced and effectively combined innovations in network architectures, software defined networking and application-defined networking.
European project NEPHELE is developing an optical network infrastructure that leverages optical switching within a software-defined networking (SDN) framework to overcome the bandwidth and energy scaling challenges of datacenter networks. NEPHELE relies on commercial off the shelf (COTS) photonic components for its data plane in order to expedite its deployment cycle while at the same time leverages open source control plane frameworks to maximize compatibility with existing infrastructures. NEPHELE introduces a dynamic optical network infrastructure for scale-out, disaggregated datacenters. NEPHELE builds on the enormous capacity of optical links and leverages hybrid optical-electronic switching to scale bandwidth at reduced cost and power compared to current technologies. In order to effectively integrate the new paradigm of optical switching into the datacenter networking ecosystem, NEPHELE followed an end-to-end approach extending from the datacenter architecture to the overlaying control plane and up to the interfaces with the application, in order to deliver a fully functional networking solution. The project brought together a multi-disciplinary workplan that introduced and effectively combined innovations in network architectures, software defined networking and application-defined networking.
During its 3-year workplan the project achieved all of its objectives:
• Objective 1: Develop scalable hybrid interconnect architecture for a dynamically reconfigurable datacentre.
A hybrid interconnect architecture was developed and studied thoroughly in terms of scalability, cost and power consumption. NEPHELE’s network architecture scales linearly with the number of datacenter hosts and consolidates compute and storage networks over a single, Ethernet optical TDMA network. Low latency, hardware-level dynamic re-configurability and quasi-deterministic Quality-of-Service (QoS) are supported in view of disaggregated datacenter deployment scenarios.The studies underpin the feasibility to scale the NEPHELE architecture at a medium-sized datacentre achieving strong reduction in power consumption, yet without a cost premium.
• Objective 2: Extend SDN control plane for slotted network operation for the dynamic allocation of optical DC network resources.
NEPHELE developed the OCEANIA and Opendaylight SDN controllers, managing and configuring the ToR and POD switches of its architecture. The north- and south-bound interfaces of the network controller were also developed along with the SDN agents, allowing the realization of slotted operation of the DCN dataplane. NEPHELE’s SDN framework is available as open-source software.
• Objective 3: Integrate application centric allocation of network resources with SDN datacenter orchestrator.
Several scheduling / bandwidth allocation algorithms were developed and evaluated during the project reducing execution time even for large DCNs. Integration of the scheduling algorithms in OCEANIA allows the dynamic reconfiguration of the data plane components according to application requirements.
• Objective 4: Enable end-to-end SLA provisioning with dynamic inter-datacenter resource management.
NEPHELE studied and evaluated the performance of resource allocation algorithms for inter-datacenter use cases. The NEPHELE inter-domain orchestrator, NIDO, interfaces i) with the OCEANIA controller for NEPHELE datacenter domains, and ii) with NEPHELE Julius SDN controller for the elastic optical transport network domain that connects NEPHELE datacenters. NIDO orchestrates through the domain controllers the provisioning of end-to-end inter-datacenter connections, providing the required service levels.
• Objective 5: Develop fully functional components enabling network convergence over a transparent Ethernet optical DC network.
NEPHELE developed and characterized the main elements of its network architecture, namely the ToR and pod switch comprising fully functional optical subsystems controlled by an Ethernet switch with FPGA port extenders. A broad range of optical components were experimentally investigated during the ToR and pod development stage. Alongside with the hybrid ToR implementation, a forward-looking all-optical design was implemented. In addition, a converged switch, storage and compute platform was developed, comprising optically enabled and interchangeable switch subsystems. This activity also led to the invention of an electro-optical connector for object-oriented end nodes.
• Objective 6: Demonstrate dynamic inter- and intra-datacenter connectivity with prototype subsystems.
The software and hardware components of NEPHELE were integrated successfully into a demo platform comprising a vertically integrated datacentre testbed assembly. Dynamic intra-datacenter operation was achieved in various routing scenarios during real time demos. The NEPHELE testbed was incorporated into the developed inter-datacenter demonstration platform verifying NEPHELE’s inter-DC connectivity and operation.
• Objective 1: Develop scalable hybrid interconnect architecture for a dynamically reconfigurable datacentre.
A hybrid interconnect architecture was developed and studied thoroughly in terms of scalability, cost and power consumption. NEPHELE’s network architecture scales linearly with the number of datacenter hosts and consolidates compute and storage networks over a single, Ethernet optical TDMA network. Low latency, hardware-level dynamic re-configurability and quasi-deterministic Quality-of-Service (QoS) are supported in view of disaggregated datacenter deployment scenarios.The studies underpin the feasibility to scale the NEPHELE architecture at a medium-sized datacentre achieving strong reduction in power consumption, yet without a cost premium.
• Objective 2: Extend SDN control plane for slotted network operation for the dynamic allocation of optical DC network resources.
NEPHELE developed the OCEANIA and Opendaylight SDN controllers, managing and configuring the ToR and POD switches of its architecture. The north- and south-bound interfaces of the network controller were also developed along with the SDN agents, allowing the realization of slotted operation of the DCN dataplane. NEPHELE’s SDN framework is available as open-source software.
• Objective 3: Integrate application centric allocation of network resources with SDN datacenter orchestrator.
Several scheduling / bandwidth allocation algorithms were developed and evaluated during the project reducing execution time even for large DCNs. Integration of the scheduling algorithms in OCEANIA allows the dynamic reconfiguration of the data plane components according to application requirements.
• Objective 4: Enable end-to-end SLA provisioning with dynamic inter-datacenter resource management.
NEPHELE studied and evaluated the performance of resource allocation algorithms for inter-datacenter use cases. The NEPHELE inter-domain orchestrator, NIDO, interfaces i) with the OCEANIA controller for NEPHELE datacenter domains, and ii) with NEPHELE Julius SDN controller for the elastic optical transport network domain that connects NEPHELE datacenters. NIDO orchestrates through the domain controllers the provisioning of end-to-end inter-datacenter connections, providing the required service levels.
• Objective 5: Develop fully functional components enabling network convergence over a transparent Ethernet optical DC network.
NEPHELE developed and characterized the main elements of its network architecture, namely the ToR and pod switch comprising fully functional optical subsystems controlled by an Ethernet switch with FPGA port extenders. A broad range of optical components were experimentally investigated during the ToR and pod development stage. Alongside with the hybrid ToR implementation, a forward-looking all-optical design was implemented. In addition, a converged switch, storage and compute platform was developed, comprising optically enabled and interchangeable switch subsystems. This activity also led to the invention of an electro-optical connector for object-oriented end nodes.
• Objective 6: Demonstrate dynamic inter- and intra-datacenter connectivity with prototype subsystems.
The software and hardware components of NEPHELE were integrated successfully into a demo platform comprising a vertically integrated datacentre testbed assembly. Dynamic intra-datacenter operation was achieved in various routing scenarios during real time demos. The NEPHELE testbed was incorporated into the developed inter-datacenter demonstration platform verifying NEPHELE’s inter-DC connectivity and operation.
NEPHELE’s vertical development approach extended from the datacenter architecture to the overlaying control plane and its interface with the application, in order to deliver a fully-functional networking solution, introducing network virtualization to the optical layer. Within its multidisciplinary workplan NEPHELE pushed the state-of-the-art with the following innovations:
-A scalable data plane architecture, leveraging mature/commercial-off-the-shelf (COTS) photonic component technologies. To enable dynamic and efficient sharing of resources, the NEPHELE network operates in a slotted time-division-multiple-access (TDMA) manner.
-An SDN control and orchestration framework capable of managing the underlying data plane elements. NEPHELE’s framework is the first to extend prominent SDN platforms with TDMA functionality, adding the capability to dynamically assign network resources directly at the optical layer. Fast resource allocation (scheduling) algorithms were developed and integrated as add-ons to the SDN platform.
NEPHELE’s wider societal implications are intertwined with the prominent effect of the internet and datacenters -serving as the internet’s main hubs- on our daily lives. The internet, as the means of accessing information, has evolved into a key driver for economy, as well as into a means for social interaction and social inclusion. Data centres are the enablers for all types of digital goods and services, along the vision for a digital single market. Even more, high-performance computing infrastructures have a significant impact in a broad variety of societal challenges, giving rise to breakthroughs in medicine, material design, climate modelling and more. NEPHELE technology aims to provide a viable solution for gracefully scaling capacity in datacenters, and can therefore underpin the societal implications of enhanced datacenter connectivity.
-A scalable data plane architecture, leveraging mature/commercial-off-the-shelf (COTS) photonic component technologies. To enable dynamic and efficient sharing of resources, the NEPHELE network operates in a slotted time-division-multiple-access (TDMA) manner.
-An SDN control and orchestration framework capable of managing the underlying data plane elements. NEPHELE’s framework is the first to extend prominent SDN platforms with TDMA functionality, adding the capability to dynamically assign network resources directly at the optical layer. Fast resource allocation (scheduling) algorithms were developed and integrated as add-ons to the SDN platform.
NEPHELE’s wider societal implications are intertwined with the prominent effect of the internet and datacenters -serving as the internet’s main hubs- on our daily lives. The internet, as the means of accessing information, has evolved into a key driver for economy, as well as into a means for social interaction and social inclusion. Data centres are the enablers for all types of digital goods and services, along the vision for a digital single market. Even more, high-performance computing infrastructures have a significant impact in a broad variety of societal challenges, giving rise to breakthroughs in medicine, material design, climate modelling and more. NEPHELE technology aims to provide a viable solution for gracefully scaling capacity in datacenters, and can therefore underpin the societal implications of enhanced datacenter connectivity.