Feature Stories - Supercomputing calms troubled waters
Turbulence can be a major problem. When water flows around a cylinder it creates little vortices in its wake, and these vortices exert a force on the cylinder creating a vibration, called a 'Vortex-induced vibration' (VIV). The impact of these VIVs depends on the relative smoothness or turbulence in the fluid; the greater the turbulence, the higher the impact of vortex-induced vibrations. But even in relatively calm conditions researchers have found evidence of VIV. The degree of smooth or turbulent flow is quantified by a measure called 'Reynolds' (Re) numbers. The higher the Reynolds number the greater the turbulent flow; a low Reynolds number means smooth flow. For example, fluids moving through a pipe typically demonstrate laminar, or smooth, predictable flow at Re 2000 or less, while turbulence begins above Re 3000, with transitional states in-between. In oil rig structures like marine riser pipes and pipelines linking to offshore oil platforms, the problem is far more significant: these structures are often exposed to Reynolds numbers greater than Re 100,000, creating a potent and damaging force. Scientists working in the Cylinder study sought to develop sufficiently detailed knowledge about turbulence and cylinder interactions to find new ways of designing pipelines which minimise the occurrence of vibration. To that end they wanted to analyse the performance of a turbulence model by comparing it with a direct computer simulation - called a 'Direct numerical simulation' (DNS) - of turbulence, at a relatively low Reynolds number. It might sound simple but it is a vast and complex task. 'Computational fluid dynamics' (CFD) is one of the most challenging problems in maths and computer science. The field is dominated by the Navier-Stokes equations, a series of formulae that adapt Newton's laws of motion to fluids. These equations are extremely complex, and even at relatively low Reynolds numbers, where there is comparatively little turbulence, problems can quickly exceed the performance of the most powerful supercomputers. 'Trying to understand these vortices represents a new approach in computational fluid dynamics. To do this, we need to include information about the full flow field. This makes the calculations very expensive to run,' remarks Roel Verstappen of the Institute of Mathematics and Computing Science at the University of Groningen. They are expensive - in supercomputing terms - because fluid flow is very complex. 'If you want to get the full solution in turbulent flow you have to solve all physical scales in the flow. As well as solving the equations at the scale of the cylinder itself, you also have very small motions which are 1,000-10,000 times smaller than the diameter of the cylinder,' Dr Verstappen explains. That is why models and modelling methods are so important. Where DEISA comes in The Cylinder study needed to access very powerful supercomputers to perform direct simulations, so it asked the 'Distributed European infrastructure for supercomputing applications' (DEISA) for help. Over the course of five years and two projects, DEISA has linked Europe's most powerful supercomputers via a network, and developed software that makes it easy for researchers to use the massive computer power that DEISA makes available. They also developed support and advice services to ensure researchers can get the greatest benefit from the available equipment. As part of its work, DEISA set up the 'DEISA extreme computing initiative' (DECI) to provide leading-edge scientific research in Europe with the best computing resources. This ensures that the most powerful resources go to the kind of research that can make the best use of them. The turbulence research done by the Cylinder study is a good example of this. Cylinder gathered scientists from the Universitat Politècnica de Catalunya, CTTC, in Spain; The Maritime Research Institute Netherlands (MARIN); and the Institute of Mathematics and Computing Science, University of Groningen in The Netherlands to run a direct numerical simulation of turbulence at Re 22,000, which is relatively low but still computationally challenging. 'DNS simulations are not feasible yet at higher Re-numbers, because the required number of floating point computations becomes simply too large,' says Dr Verstappen. Thanks to DEISA, Dr Verstappen and his colleagues were able to carry out the full computation, and the calculations showed very good agreement between the modelling method and DNS. The test required around 650,000 hours of computing time. 'It sounds like a lot, but it isn't really,' says Dr Verstappen. 'If we had considered a more complex turbulent flow it could easily have required 10 or 20 or even a 100 million hours. If you want to carry out the kind of research we've been doing there is only one source of computing resources you can turn to - and that is DEISA. At the national level, 1 million CPU hours is very difficult to obtain, especially every year. DEISA made the Cylinder project possible,' he notes. The computations of the Cylinder research were performed at Barcelona Supercomputer Centre. There were a few test runs, which took about 50,000 CPU-hours. Then 600,000 hours were invested on the main run. The code was developed at UPC - Barcelona TECH and University of Groningen. Local machines performed visualisation and data analysis, thanks to a new visualisation tool, developed in the Scientific Visualisation and Computer Graphics Group at University of Groningen. Ultimately, the DECI research carried out by the Cylinder project aims to improve rig design. Currently designers tackle the problem of oilrig and submarine pipelines by carrying out experiments using physical models set up in tanks of water. After generating vibrations in the models they measure the effects and use these results to try to calculate the strength of materials needed for cylindrical structures. Cylinder's work, however, may soon allow engineers to use mathematical simulations to gather the data they need instead, saving time and money. Moreover, the work will have the potential for wide application in a number of fields. 'The results of the Cylinder project,' says Dr Verstappen, 'are enabling us to develop simpler models that can be used in computations for engineering applications. This flow simulation forms the first step towards accurate numerical simulations of vortex-induced vibrations of marine riser pipes,' he adds. Cylinder was carried out in 2009. As a follow-up, and with support from the Dutch Maritime Innovation Platform (MIP), the group have gone on to devise a number of simplified models. These will then be tested to determine which is the most accurate. 'These simpler models will allow designers to use local parallel computing facilities to assist in their designs,' notes Dr Verstappen. 'Typically, designers require many computer runs in the design phase. But, as a result of the Cylinder work and the follow-up project, designers will not only require fewer runs; they will be able to carry them out using simpler computing facilities.' DEISA2 was funded to the tune of EUR 10.24 million (of EUR 18.65 million total project budget) under the EU's Seventh Framework Programme for research, 'e-Science grid infrastructures' sub-programme. Useful links: - 'Distributed European infrastructure for supercomputing applications' - DEISA2 project data record on CORDIS - e-Infrastructures programme / projects - Cylinder study Related articles: - Linking supercomputers to simulate the sun, the climate and the human body - Supercomputers target HIV - Climate models run supercomputer catwalk - Supercomputing gets its own superhero - The grid: a new way of doing science - Europe's fusion researchers to tap into top supercomputing resources