Final Report Summary - PICASSO (imProved reliabIlity inspeCtion of Aeronautic structure through Simulation Supported POD)
Executive Summary:
The aim of PICASSO (imProved reliabIlity inspeCtion of Aeronautic structure through Simulation Supported POD) was to build a new and original concept of “simulation supported Probability of Detection (POD) curves based on Non Destructive Testing (NDT) simulation in addition to existing experimental data base”.
The main objectives of PICASSO were to have more accurate and reliable POD curves in order to get appropriate damage tolerance assessments for critical parts to address the potential risk of failure from induced anomalies within the Approved Life the part, and to overcome cost issue of the POD campaign with Non Destructive Testing simulation techniques.
Thanks to the PICASSO project, the Non Destructive Testing simulation softwares have been highly improved:
• new tools are available and were implemented (3D tools, CAD file interpretation, parameters fluctuation scenary ...);
• the models are now able to consider some phenomena induced by the complexity of some materials properties and are thus now more accurate to predict the influence of the metallurgical structure on the inspection performances;
• several new models developments have been successfully performed to support the simulation of the inspections of the validations cases defined.
Regarding the Probability of Detection (POD) issues and the introduction of simulated data in POD studies :
• the uncertainties modelling strategy have been defined and the Design Of numerical Experiment (DoE) tools have been prototyped;
• the transfer function approach has been imagined to calculate a POD on a configuration B using the data of a configuration A (cousin) and simulation of the A and B configurations;
• alternative non-parametric approaches have been developed to deal with the cases that cannot be dealt with parametric methods (classical Berens models)
• developments of analysis tools aiming at checking the consistency of the input data towards these hypotheses have therefore been carried out;
• finally, the project’s library in C++ useable by all the NDT software of the project have been developed;
The assessment of the PICASSO new POD methodology has been made using validations cases. These validation cases were selected by the industrial partners in order to be representative of the main problematics they have to face when using NDT simulation softwares. These validation cases have been manufactured. For these validation cases several NDT laboratories have carried out the experimental NDT investigations. Furthermore simulations of these experiments have been done. Based on these experimental and simulated data the POD curves have been calculated. The results have been compared and the equivalency have been described. For some validation cases, both POD calculations agreed very well. For every NDT method investigated, ET, RT and UT, at least one validation case could be successfully carried out. For every method the measured and simulated data agreed very well and it could be shown that simulation of NDT methods could be used to calculate POD curves with sufficient confidence.
The main objectives of the project are fulfilled.
A recommendation for simulation based POD calculation was derived and a final public Workshop has been held.
The concept of “simulation supported POD” has thus been validated on some first realistic results.
Its intensive use and the building of a future European standard on simulation supported POD will be the next step towards the entry into service of such tools.
Project Context and Objectives:
The aim of PICASSO (imProved reliabIlity inspeCtion of Aeronautic structure through Simulation Supported POD) is to build a new and original concept of “simulation supported Probability of Detection (POD) curves based on Non Destructive Testing (NDT) simulation in addition to existing experimental data base”.
Technical context
Most aeronautical components have to be designed on damage tolerance concept. The damage tolerance design assumes that aeronautical structures contain flaws. This means that the potential existence of component imperfections as the result of inherent material structure, material processing, component design, manufacturing or usage is recognized and this situation is addressed through the incorporation of fracture resistant design, process control and Non-Destructive Testing (NDT) inspections. Therefore, periodic maintenance intervals are defined by the design office on the base of maximum size of defect potentially present in the component and material properties (crack growth rate).
This process requires to perfectly know the minimum size of indication that can be reliably detected during manufacturing and in-service inspections. This information is obtained through Probability of Detection (POD) campaigns (see MIL-HDBK-1823A for recommendations concerning POD determination). A POD campaign consists in applying the NDT procedure associated to the part by several NDT operators on a large panel of samples with and without defects representative of possible damage on the part. Statistical studies are performed on NDT results. The objective is to determine the curve which calculates the probability to detect a defect type in the proposed configuration (probability vs. defect size).
Innovation context
Today’s conventional methodology for POD campaigns is too much cost and time consuming and configurations equivalence, which could avoid new POD campaigns, are not available. In the framework of PICASSO project, the benefits of an innovative concept will be evaluated. This concept is the “simulation supported POD based on using NDT simulation” tools for future POD determinations.
The simulation supported POD approach constitutes very promising solution for the evaluation of aeronautic NDT inspection. By incorporating simulated data to the experimental statistical campaigns it becomes possible to reduce the cost and time while increasing the reliability of POD evaluations. The use of quantitative and validated numerical models will allow investigating more representative samples population than the usual purely experimental POD campaigns, to cross more completely the effects of the different influential parameters and to reduce the a priori hypothesis done today on the statistical models.
Objectives
The objectives of PICASSO were :
• to have more accurate and reliable POD curves in order to get appropriate damage tolerance assessments for critical parts to address the potential risk of failure from induced anomalies (from material, manufacturing or service) within the Approved Life the part,
• to overcome cost issue of the Probability of Detection campaign with Non Destructive Testing simulation techniques.
PICASSO developed the methodology and the corresponding software tools to implement the concept of simulation supported POD for the engine aircraft industry.
The PICASSO technical objectives were consequently :
• To determine reliable and realistic initial input for the simulation softwares
Reliable NDT simulations models are based on initial input depending on material properties (noise, grain size, anisotropy, ..), defect description (cracks, inclusions, …), and data from equipment (sensors, detectors). The initial real conditions of the component must be precisely known to determine representative modelling inputs. The results of the NDT inspections may be strongly affected in function of the material characteristics (grain size, noise, anisotropy…), the real shape and knowledge of the (un)expected defects, and data from equipment.
• To improve the NDT simulation software so that they will be more accurate, reliable and numerically efficient
The exploitation of simulation for assisting the determination of POD curves require physical models able to account for the importance of the different influential parameters and their possible fluctuations. The inspected parts being generally of complex geometry and designed by CAD specific developments have to be done in order to make the simulation tools able to efficiently handle the CAD geometries and their variations as an input of POD studies. Another important issue to be addressed is the modelling of the phenomena induced by the complexity of materials constituting the inspected parts. This concerns mainly, for ultrasonic and electromagnetic techniques, prediction of the noise arising from the metallurgical structure and for X-rays, modelling of the diffraction phenomena on large single crystals.
• To develop a POD methodology
The simulation supported POD approach constitutes an innovative and very promising solution for the evaluation of aeronautic NDT inspection. By incorporating simulated data to the statistical campaigns it becomes possible to drastically reduce the cost and time while increasing the reliability of POD evaluations. The use of quantitative and validated numerical models will allow investigating more representative samples population than the usual purely experimental POD campaigns, to cross more completely the effects of the different influential parameters and to reduce the a priori hypothesis done today on the statistical models. To put in practice this low cost efficient POD approach requires the definition of optimized procedures for combining experimental and numerical data which will be addressed during the project which should result in the establishment of common European recommendations
• To Assess and validate the concept
The simulation supported POD methodology developed in the framework of PICASSO will be validated by comparison with experimental POD data.
To achieve these objectives, the PICASSO project were organised in six work packages including one work package dedicated to the management and the coordination (WP5) and one to the exploitation and dissemination of the projects results (WP6).
WP1 : Initial input
Global objectives:
WP1 had the following objectives:
• Selection of validation cases
• Support of WP2 “Modelling” with the description of material properties, actual crack and volumetric defect geometries in CAD form and equipment.
• Support of WP3 “Simulation supported POD” and WP4 ‘’Assessment of the new POD methodology’’ with the definition of samples which be used in direct comparison of the traditional and model based POD calculations.
Correlations with other WPs:
The material characterisations obtained as result of the WP1 ‘initial input’ were the starting input for the materials models developed in WP2. The characterisations lead in WP1 provided all the experimental material properties to allow the development of the material models (material and defect characterisation, equipment data concerning UT, RT and ET techniques).
WP2: Modelling
Global objectives:
The simulation supported POD approach is based on the application of quantitative models which will take into account the complexity of real situations. In particular, models have to be able to consider the phenomena induced by the complexity of parts and materials (texture, microstructure, structural noise …), the complexity of geometries, of defects, and the fluctuations coming from a large number of influential parameters. Models were highly improved by developers to achieve accurate, reliable and numerically efficient simulation tools. These models were used by industrial partners to verify if the specificity of aeronautical metallic parts providing sensitivity of inspection was taken into account and if models were reliable enough for an intensive use for POD calculations.
Correlations with other WPs:
The material characterisations obtained as result of the WP1 ‘initial input’ were the starting input for the materials models developed in WP2. The characterisations lead in WP1 provided all the experimental material properties to allow the development of the material models (material and defect characterisation, equipment data concerning UT, RT and ET techniques).
The simulation results obtained through the development of the models (material models and current NDT system models) and deliverables of WP2 were reliable enough to be introduced as input for the POD models WP3. The reliability of the simulation has been ensured by comparison between experimental and modelling on specific test cases also performed in WP2.
WP3: Simulation supported POD
Global objectives:
WP3 was the core of the PICASSO project since it focused on Probability of Detection (POD) issues and dealt with the introduction of simulated data in POD studies. The “design lighter” goal implies reliable determination of POD curves when design is based on damage tolerance, as it is the case in aeronautics. This WP objective were to develop a simulation supported POD methodology within a European framework.
The goal of WP3 were to provide common methodologies and tools for low cost POD determination using a simulation supported strategy. Major expected result were the development of a prototype POD software platform.
Correlations with other WPs:
WP3 took input from WP1 “Initial inputs” through the tasks “Material properties” (T1.2) “Defect characterization” (T1.3) and “Data from equipment” (T1.4) that provided information on test samples material, defects, and equipment for the POD studies.
Inputs from WP2 “Modelling” provided tools for the development of methodology to generate simulated data including uncertainties (Design of numerical experiment).
WP3 provided a simulation POD software platform, which were validated in WP4.
WP4: Assessment of the new methodology
Global objectives:
WP4 had 4 objectives:
• Production of POD data
• Calculation and comparison of POD curves, conventional and simulation supported
• Capability criteria for simulation supported POD curves
• Recommendations and training curses
The goal of WP4 was to assess and validate the new simulation supported POD methodology developed in the framework of PICASSO (WP3). It was performed by the comparison between experimental and simulation POD data. These data were collected or generated in the frame of WP4. Major expected result was the validation of the simulation supported POD methodology and the establishment of recommendations and training for the use of the platform.
Correlations with other WPs:
WP4 took input from WP1 to identify the validation cases selected by the PICASSO partners.
The generation of simulation supported POD data was performed by using the simulation POD software platform developed in WP3.
Project Results:
The simulation supported POD approach is based on the application of quantitative models which have to take into account the complexity of real situations. In particular, models have to be able to consider the phenomena induced by the complexity of parts and materials (texture, microstructure, structural noise …), the complexity of geometries, of defects, the variations in the characteristics of the equipments and the fluctuations coming from a large number of influential parameters.
The first steps of PICASSO were thus to obtain these inputs that the models have to be developed against.
• Some target material properties such as UT noise and RT diffraction have been fully characterized :
Material properties for Titanium alloys and the diffraction phenomena of a single crystal Ni were described.
• The characteristics of the major equipments used for the NDT of aeronautical parts (ET probes, UT transducers, digital RT) have been established :
Eddy Current probes, which were chosen to represent the main coil arrangements used for eddy current testing of aerospace components, were characterized using x-ray computed tomography enabling to image the coils and obtain the necessary measurements for accurate modelling inspection.
A new digital RT system, modified and adapt for relevant aeronautical components, was up and running in 2011. The system has been fully characterised and validated.
Regarding UT : conventional, phased array and multizone probes were characterized.
All these data were provided to the softwares modellers as inputs for their developments.
Furthermore, before PICASSO, the models had some limitations which did not lead us to simulate some experimental configurations. The industrial partners have thus selected some case studies (called validation cases) which were representative of the main problematics they have to face when using NDT simulation softwares. The three targeted NDT methods (RT, ET and UT) were covered.
The Non Destructive Testing simulation softwares have been highly improved in the framework of PICASSO :
• New tools are available and were implemented (3D tools, CAD file interpretation, parameters fluctuation scenary ...)
• The models are now able to consider some phenomena induced by the complexity of some materials properties and are thus now more accurate to predict the influence of the metallurgical structure on the inspection performances
• Several new models developments have been successfully performed to support the simulation of the inspections of the validations cases defined.
Another major objective of PICASSO was to focus on Probability of Detection (POD) issues and deal with the introduction of simulated data in POD studies.
To achieve these goals, the first activity was to focus on the uncertainties modelling strategy. The influent NDT parameters whose variations have a strong impact on the result were defined and the statistical laws for these parameters variation (Design of numerical experiment – DoE) were determined. The DoE tools have been prototyped and the corresponding software modules are available.
The second activity was to work on the combination of experimental and modelling data and to develop a methodology for the transfer function POD approach. A transfer function approach has been proposed to obtain POD curve in Alumium based on simulation data in Titanium and Aluminium. Results were evaluated on a ET case provided by EADS (HFET on fatigue cracks). This new approach shows how linear methods can be extended to account for piecewise linear relationships between signal and crack length. The new methodology requires linear piecewise signal fitting. An heuristic method has been proposed to compute a piecewise/mixture POD. The proposed methodology to generalize the transfer function approach to piecewise linear relationships between signal and crack length requires the following steps :
• Step 1: to estimate parameters of the piecewise linear regressions for simulated data for Titanium and Aluminum and for experimental data (fatigue cracks) for Titanium,
• Step 2: to collect the three associated breakpoints denoted , , and thus to define the four resulting intervals on which computing the predicted piecewise mean transferred signal for Aluminum,
• Step 3: to build and to apply the piecewise transfer function to each of the intervals obtained at step 2,
• Step 4: to compute a POD curve for the detection of fatigue cracks in Aluminum.
The third activity was the improvement of the POD estimation models. Classical Berens models (parametric) for estimation have been analysed and limits and hypotheses of the models have been highlighted so that to orientate developments on alternative approaches.
Alternative non-parametric approaches have been developed (Quartile regression and kernel regression) to deal with the cases that cannot be dealt with parametric methods.
Furthermore, developments of analysis tools aiming at checking the consistency of the input data towards these hypotheses have therefore been carried out. These tools include :
• Plots of residual graphs (difference between actual data and regression fit);
• Quantitative evaluation of the noise;
• Alternative data representation for evaluation of the regression fit.
The prototype implementation of specified methods have been made.
Finally, the project’s library in C++ useable by all the NDT software of the project have been developed.
The assessment of the PICASSO new POD methodology has been made using the validations cases. These validation cases have been manufactured. For these validation cases several NDT laboratories have carried out the experimental NDT investigations. Furthermore simulations of these experiments have been done. Based on these experimental and simulated data the POD curves have been calculated. The results have been compared and the equivalency have been described. For some validation cases, both POD calculations agreed very well. For every NDT method investigated, ET, RT and UT, at least one validation case could be successfully carried out. For every method the measured and simulated data agreed very well and it could be shown that simulation of NDT methods could be used to calculate POD curves with sufficient confidence.
• Evaluation of the POD calculation methodology for ET POD calculation
Simulation-based POD curves have been determined and compared to experimental POD curves for an application case consisting of High Frequency Eddy Currents Testing of fatigue cracks in flat Titanium parts. The simulation results have been obtained using the POD module proposed in the CIVA software. Simulation-based POD results are in good agreement with experimental POD for this configuration. It is shown that by taking into account a limited number of variability sources (start scan position, crack high, probe angle and electrical contacts on cracks), a simulation-based POD curve with features very similar to the experimental curve is obtained. From this successful attempt, the basic methodology and principle is considered with confidence.
An approach of confidence evaluation for POD using Uncertainty Propagation through simulation codes has been proposed and demonstrated on two different NDT application cases. The so provided confidence has a different meaning than the classical confidence estimation based on the sample size but has the same objective - integrate quantitatively the level of available knowledge in the process description – and results in a robust process to determine POD curves using uncertainty propagation through simulation.
• Evaluation of the POD calculation methodology for RT POD calculation
With the help of wedges and sheet metal with drilled holes, the POD of pores in a microradiograph has been measured experimentally. The whole configuration has been simulated successfully using aRTist including the image evaluation by the human operator. For small indications with good experimental support, an excellent agreement of the POD within a few percent has been found. For larger indications, good qualitative agreement is observed. Adjustments have been done to the simulation software to closely match the experimental data, which readily lead to a simulation supported POD methodology.
The proposed simulation supported POD methodology for RT has been successfully tested.
• Evaluation of the POD calculation methodology for UT POD calculation
The recommended general method for performing a simulated supported UT POD study was developed.
Using a combination of empirical and simulated POD data has several advantages, if only to improve the confidence that simulated approach has correctly replicated the inspection scenario. The number of real defects will depend on the requirements and reason choosing a combined approach.
The main objective of the project is fulfilled.
A recommendation for simulation based POD calculation was derived and a final public Workshop has been held.
The concept of “simulation supported POD” has thus been validated on some first realistic results.
Potential Impact:
To cover this question, a questionnaire was sent and filled by the partners (milestone M6.2). A summary of the answers is proposed here-under :
1. Which are the target “customers” of the concept of Simulation Supported POD developed in the framework of PICASSO ?
Any company, from any industrial sector, involved in concerns about NDE methods reliability with reduced costs, to ensure appropriate answers to customers (request for justification of proposed procedures) or to their own needs (new methods, modification of existing methods due to evolution of requirements or adaptation to materials and systems – eg removal of existing probes or devices…)
2. List the main Strengths of the concept of Simulation Supported POD developed in the framework of PICASSO ?
The simulated POD approach developed by PICASSO enables more reliable and low cost POD studies to be carried out. It will also enable quantitative reliability assessments of NDT techniques when an empirical study is not practical or not possible.
Increase knowledge of NDE procedures, limitations and influencing factors and of the underlying physics This understanding is likely to lead to improved inspection reliability.
It enable also fast technique capability estimations and a decrease the need for experimental test pieces
3. List the main Weaknesses of the concept of Simulation Supported POD developed in the framework of PICASSO ?
Difficulties arise in determining all input data relevant for simulation and full exploitation of acquired data.
It is not always easy to achieve the fact that :
• A validated model is required.
• A thorough understanding of factors that influence the inspection is required.
It could be hard to justify the use of only simulated data in order to estimate NDE procedures because of the difficulties to model the real complex flaw geometries and some procedure parameters may be difficult to model and must be justified experimentally.
4. Which elements of the PICASSO results / methodologies need strengthening in order to become more exploitable ?
Currently it is not exploitable for all inspections. Through the PICASSO project the capability of NDT models has increased. The simulated POD approach could be used for more inspections if NDT models were further improved. Also, a recognized method for validating NDT models would be useful.
A more comprehensive validation of the different models (in UT,ET,RT) would be necessary prior to their use for simulation supported POD.
5. What obstacles or difficulties might you face when trying to exploit the PICASSO results / methodologies ?
From a technical point of view, determination of input data related to uncertainties sources and their related distributions and margins remain difficult to establish and may cause discrepancies between POD based on simulated data and POD based on experimental data, as well as bias over mixed POD data. Other potential difficulties arise when combining several codes and softwares (exploitation of data through different formats, etc…) .
The main issue is the understanding of factors that influence the inspection reliability. The PICASSO project developed methods to carry out a study to do this. However, understanding how well these factors are understood can be challenging. Members of the PICASSO group are continuing to work on this issue.
6. Please list all the opportunities and plans that your organization has for the exploitation of the PICASSO results / methodologies
PICASSO has increased the quality and reliability of POD estimations, simplified planning of POD studies, increased NDE knowledge and understanding of POD and increase the use of simulation tools for inspection of engine components.
Academic partners :
• further development of RT simulation tool
• use/distribution of POD tools
• supervision of doctoral students
• presentation of results at 5th European-American Workshop on Reliability of NDE, Berlin, 2013
• exploitation of the developments within the next version (commercial or not) of the simulation softwares which are to be released soon.
Industrial partners :
• incorporation of the methods developed in PICASSO in the industries’s procedures to determining the reliability of its inspections.
• ability to carry out simulated POD studies is a key part of the industries NDT strategy.
• support for the damage tolerance design
• evaluation of the absolute performance of ET, UT and RT method in the framework of critical parts inspection
• presentation of results at 5th European-American Workshop on Reliability of NDE, Berlin, 2013
7. Please describe what could be the plan to bring the PICASSO technology to higher TRL (6-7)
The techniques developed by PICASSO require further use to gain improved knowledge of how to implement simulated POD methods.
Members of the PICASSO group are currently working on a complementary document describing the recommended methods for carrying out simulated POD studies. The aim will be to capture the knowledge gained during the PICASSO project and have an agreed method that can be tested by the industrial partners. It is hoped that this will lead to common accepted methods.
The importance of the work can be measured by the fact that the PICASSO group members are continuing to work (after the project is finished) towards a common method that will allow the knowledge gained through the project to be used by industry.
Furthermore, it is already planned to continue and reinforce collaborations with other groups such as the MAPOD group in USA for an improved leverage about the overall use of simulation and dedicated tools (such as simulation supported POD) in industrial sectors. Such a collaboration with the MAPOD group has already been intiated by the PICASSO partners. Also, needs for better acceptance of simulation by standardization /certification organizations. To address this point, it is also planned to continue the communication with the RISK Group : PICASSO results will be presented during their next European meeting in June 2013.
List of Websites:
http://www.picasso-ndt.eu
Relevant contact details :
Snecma (Coordinator) : MALEO Nancy - +33 (0)1 60 59 91 12 - nancy.maleo@snecma.fr
BAM (Point of Contact for aRTist software) : BELLON Carsten - + 49 - 30 - 8104 3658 - carsten.bellon@bam.de
CEA (Point o Contact for CIVA software) : MAHAUT Steeve - +33 (0)1 69 08 47 14 - steve.mahaut@cea.fr
Chalmers University (Point of contact) : WIRDELIUS Hakan - +46(0)31772 27 12 - hakan.wirdelius@chalmers.se
EADS IW (Point of contact) : DOMINGUEZ Nicolas - 33 (0)5 61 16 88 13 - Nicolas.Dominguez@eads.net
HTS (Point of contact for VIC3D software) : NADLER Michael - + 49 (0) 3523 77560 - michael.nadler@htsdd.de
IZFP-D (Point of contact for EFIT Software) : SCHUBERT Franck - +49 (0) 351 / 88815-523 - Frank.Schubert@izfp-d.fraunhofer.de
MTU (Point of contact) : BARON Hans-Uwe - +49 (0)89 14 89-26 46 - hans-uwe.Baron@muc.mtu.de
PHIMECA (Point of contact for the new POD tools) - YALAMAS Thierry - +33 1 58 51 18 02 - yalamas@phimeca.com
Rolls-Royce plc (Point of contact) : BARDEN Tim - +44 (0) 117 9797143 - Tim.Barden@Rolls-Royce.com
TURBOMECA (Point of contact) : JULIAC Etienne - +33 (0)5 59 12 10 47 - Etienne.juliac@turbomeca.fr
TWI (Point of contact) : WALLACE Philip - +44 (0)1639 873100 - philip.wallace@twi.co.uk
GKN (Point of contact for GKN software) : ROSELL Anders - +46 (0)520 94080 - anders.ar.rosell@gknaerospace.com
Technic Control (Point of contact) : LYSZCZYK Pawel - + 48 605 081 489 - p.lyszczyk@technic-control.com.pl
The aim of PICASSO (imProved reliabIlity inspeCtion of Aeronautic structure through Simulation Supported POD) was to build a new and original concept of “simulation supported Probability of Detection (POD) curves based on Non Destructive Testing (NDT) simulation in addition to existing experimental data base”.
The main objectives of PICASSO were to have more accurate and reliable POD curves in order to get appropriate damage tolerance assessments for critical parts to address the potential risk of failure from induced anomalies within the Approved Life the part, and to overcome cost issue of the POD campaign with Non Destructive Testing simulation techniques.
Thanks to the PICASSO project, the Non Destructive Testing simulation softwares have been highly improved:
• new tools are available and were implemented (3D tools, CAD file interpretation, parameters fluctuation scenary ...);
• the models are now able to consider some phenomena induced by the complexity of some materials properties and are thus now more accurate to predict the influence of the metallurgical structure on the inspection performances;
• several new models developments have been successfully performed to support the simulation of the inspections of the validations cases defined.
Regarding the Probability of Detection (POD) issues and the introduction of simulated data in POD studies :
• the uncertainties modelling strategy have been defined and the Design Of numerical Experiment (DoE) tools have been prototyped;
• the transfer function approach has been imagined to calculate a POD on a configuration B using the data of a configuration A (cousin) and simulation of the A and B configurations;
• alternative non-parametric approaches have been developed to deal with the cases that cannot be dealt with parametric methods (classical Berens models)
• developments of analysis tools aiming at checking the consistency of the input data towards these hypotheses have therefore been carried out;
• finally, the project’s library in C++ useable by all the NDT software of the project have been developed;
The assessment of the PICASSO new POD methodology has been made using validations cases. These validation cases were selected by the industrial partners in order to be representative of the main problematics they have to face when using NDT simulation softwares. These validation cases have been manufactured. For these validation cases several NDT laboratories have carried out the experimental NDT investigations. Furthermore simulations of these experiments have been done. Based on these experimental and simulated data the POD curves have been calculated. The results have been compared and the equivalency have been described. For some validation cases, both POD calculations agreed very well. For every NDT method investigated, ET, RT and UT, at least one validation case could be successfully carried out. For every method the measured and simulated data agreed very well and it could be shown that simulation of NDT methods could be used to calculate POD curves with sufficient confidence.
The main objectives of the project are fulfilled.
A recommendation for simulation based POD calculation was derived and a final public Workshop has been held.
The concept of “simulation supported POD” has thus been validated on some first realistic results.
Its intensive use and the building of a future European standard on simulation supported POD will be the next step towards the entry into service of such tools.
Project Context and Objectives:
The aim of PICASSO (imProved reliabIlity inspeCtion of Aeronautic structure through Simulation Supported POD) is to build a new and original concept of “simulation supported Probability of Detection (POD) curves based on Non Destructive Testing (NDT) simulation in addition to existing experimental data base”.
Technical context
Most aeronautical components have to be designed on damage tolerance concept. The damage tolerance design assumes that aeronautical structures contain flaws. This means that the potential existence of component imperfections as the result of inherent material structure, material processing, component design, manufacturing or usage is recognized and this situation is addressed through the incorporation of fracture resistant design, process control and Non-Destructive Testing (NDT) inspections. Therefore, periodic maintenance intervals are defined by the design office on the base of maximum size of defect potentially present in the component and material properties (crack growth rate).
This process requires to perfectly know the minimum size of indication that can be reliably detected during manufacturing and in-service inspections. This information is obtained through Probability of Detection (POD) campaigns (see MIL-HDBK-1823A for recommendations concerning POD determination). A POD campaign consists in applying the NDT procedure associated to the part by several NDT operators on a large panel of samples with and without defects representative of possible damage on the part. Statistical studies are performed on NDT results. The objective is to determine the curve which calculates the probability to detect a defect type in the proposed configuration (probability vs. defect size).
Innovation context
Today’s conventional methodology for POD campaigns is too much cost and time consuming and configurations equivalence, which could avoid new POD campaigns, are not available. In the framework of PICASSO project, the benefits of an innovative concept will be evaluated. This concept is the “simulation supported POD based on using NDT simulation” tools for future POD determinations.
The simulation supported POD approach constitutes very promising solution for the evaluation of aeronautic NDT inspection. By incorporating simulated data to the experimental statistical campaigns it becomes possible to reduce the cost and time while increasing the reliability of POD evaluations. The use of quantitative and validated numerical models will allow investigating more representative samples population than the usual purely experimental POD campaigns, to cross more completely the effects of the different influential parameters and to reduce the a priori hypothesis done today on the statistical models.
Objectives
The objectives of PICASSO were :
• to have more accurate and reliable POD curves in order to get appropriate damage tolerance assessments for critical parts to address the potential risk of failure from induced anomalies (from material, manufacturing or service) within the Approved Life the part,
• to overcome cost issue of the Probability of Detection campaign with Non Destructive Testing simulation techniques.
PICASSO developed the methodology and the corresponding software tools to implement the concept of simulation supported POD for the engine aircraft industry.
The PICASSO technical objectives were consequently :
• To determine reliable and realistic initial input for the simulation softwares
Reliable NDT simulations models are based on initial input depending on material properties (noise, grain size, anisotropy, ..), defect description (cracks, inclusions, …), and data from equipment (sensors, detectors). The initial real conditions of the component must be precisely known to determine representative modelling inputs. The results of the NDT inspections may be strongly affected in function of the material characteristics (grain size, noise, anisotropy…), the real shape and knowledge of the (un)expected defects, and data from equipment.
• To improve the NDT simulation software so that they will be more accurate, reliable and numerically efficient
The exploitation of simulation for assisting the determination of POD curves require physical models able to account for the importance of the different influential parameters and their possible fluctuations. The inspected parts being generally of complex geometry and designed by CAD specific developments have to be done in order to make the simulation tools able to efficiently handle the CAD geometries and their variations as an input of POD studies. Another important issue to be addressed is the modelling of the phenomena induced by the complexity of materials constituting the inspected parts. This concerns mainly, for ultrasonic and electromagnetic techniques, prediction of the noise arising from the metallurgical structure and for X-rays, modelling of the diffraction phenomena on large single crystals.
• To develop a POD methodology
The simulation supported POD approach constitutes an innovative and very promising solution for the evaluation of aeronautic NDT inspection. By incorporating simulated data to the statistical campaigns it becomes possible to drastically reduce the cost and time while increasing the reliability of POD evaluations. The use of quantitative and validated numerical models will allow investigating more representative samples population than the usual purely experimental POD campaigns, to cross more completely the effects of the different influential parameters and to reduce the a priori hypothesis done today on the statistical models. To put in practice this low cost efficient POD approach requires the definition of optimized procedures for combining experimental and numerical data which will be addressed during the project which should result in the establishment of common European recommendations
• To Assess and validate the concept
The simulation supported POD methodology developed in the framework of PICASSO will be validated by comparison with experimental POD data.
To achieve these objectives, the PICASSO project were organised in six work packages including one work package dedicated to the management and the coordination (WP5) and one to the exploitation and dissemination of the projects results (WP6).
WP1 : Initial input
Global objectives:
WP1 had the following objectives:
• Selection of validation cases
• Support of WP2 “Modelling” with the description of material properties, actual crack and volumetric defect geometries in CAD form and equipment.
• Support of WP3 “Simulation supported POD” and WP4 ‘’Assessment of the new POD methodology’’ with the definition of samples which be used in direct comparison of the traditional and model based POD calculations.
Correlations with other WPs:
The material characterisations obtained as result of the WP1 ‘initial input’ were the starting input for the materials models developed in WP2. The characterisations lead in WP1 provided all the experimental material properties to allow the development of the material models (material and defect characterisation, equipment data concerning UT, RT and ET techniques).
WP2: Modelling
Global objectives:
The simulation supported POD approach is based on the application of quantitative models which will take into account the complexity of real situations. In particular, models have to be able to consider the phenomena induced by the complexity of parts and materials (texture, microstructure, structural noise …), the complexity of geometries, of defects, and the fluctuations coming from a large number of influential parameters. Models were highly improved by developers to achieve accurate, reliable and numerically efficient simulation tools. These models were used by industrial partners to verify if the specificity of aeronautical metallic parts providing sensitivity of inspection was taken into account and if models were reliable enough for an intensive use for POD calculations.
Correlations with other WPs:
The material characterisations obtained as result of the WP1 ‘initial input’ were the starting input for the materials models developed in WP2. The characterisations lead in WP1 provided all the experimental material properties to allow the development of the material models (material and defect characterisation, equipment data concerning UT, RT and ET techniques).
The simulation results obtained through the development of the models (material models and current NDT system models) and deliverables of WP2 were reliable enough to be introduced as input for the POD models WP3. The reliability of the simulation has been ensured by comparison between experimental and modelling on specific test cases also performed in WP2.
WP3: Simulation supported POD
Global objectives:
WP3 was the core of the PICASSO project since it focused on Probability of Detection (POD) issues and dealt with the introduction of simulated data in POD studies. The “design lighter” goal implies reliable determination of POD curves when design is based on damage tolerance, as it is the case in aeronautics. This WP objective were to develop a simulation supported POD methodology within a European framework.
The goal of WP3 were to provide common methodologies and tools for low cost POD determination using a simulation supported strategy. Major expected result were the development of a prototype POD software platform.
Correlations with other WPs:
WP3 took input from WP1 “Initial inputs” through the tasks “Material properties” (T1.2) “Defect characterization” (T1.3) and “Data from equipment” (T1.4) that provided information on test samples material, defects, and equipment for the POD studies.
Inputs from WP2 “Modelling” provided tools for the development of methodology to generate simulated data including uncertainties (Design of numerical experiment).
WP3 provided a simulation POD software platform, which were validated in WP4.
WP4: Assessment of the new methodology
Global objectives:
WP4 had 4 objectives:
• Production of POD data
• Calculation and comparison of POD curves, conventional and simulation supported
• Capability criteria for simulation supported POD curves
• Recommendations and training curses
The goal of WP4 was to assess and validate the new simulation supported POD methodology developed in the framework of PICASSO (WP3). It was performed by the comparison between experimental and simulation POD data. These data were collected or generated in the frame of WP4. Major expected result was the validation of the simulation supported POD methodology and the establishment of recommendations and training for the use of the platform.
Correlations with other WPs:
WP4 took input from WP1 to identify the validation cases selected by the PICASSO partners.
The generation of simulation supported POD data was performed by using the simulation POD software platform developed in WP3.
Project Results:
The simulation supported POD approach is based on the application of quantitative models which have to take into account the complexity of real situations. In particular, models have to be able to consider the phenomena induced by the complexity of parts and materials (texture, microstructure, structural noise …), the complexity of geometries, of defects, the variations in the characteristics of the equipments and the fluctuations coming from a large number of influential parameters.
The first steps of PICASSO were thus to obtain these inputs that the models have to be developed against.
• Some target material properties such as UT noise and RT diffraction have been fully characterized :
Material properties for Titanium alloys and the diffraction phenomena of a single crystal Ni were described.
• The characteristics of the major equipments used for the NDT of aeronautical parts (ET probes, UT transducers, digital RT) have been established :
Eddy Current probes, which were chosen to represent the main coil arrangements used for eddy current testing of aerospace components, were characterized using x-ray computed tomography enabling to image the coils and obtain the necessary measurements for accurate modelling inspection.
A new digital RT system, modified and adapt for relevant aeronautical components, was up and running in 2011. The system has been fully characterised and validated.
Regarding UT : conventional, phased array and multizone probes were characterized.
All these data were provided to the softwares modellers as inputs for their developments.
Furthermore, before PICASSO, the models had some limitations which did not lead us to simulate some experimental configurations. The industrial partners have thus selected some case studies (called validation cases) which were representative of the main problematics they have to face when using NDT simulation softwares. The three targeted NDT methods (RT, ET and UT) were covered.
The Non Destructive Testing simulation softwares have been highly improved in the framework of PICASSO :
• New tools are available and were implemented (3D tools, CAD file interpretation, parameters fluctuation scenary ...)
• The models are now able to consider some phenomena induced by the complexity of some materials properties and are thus now more accurate to predict the influence of the metallurgical structure on the inspection performances
• Several new models developments have been successfully performed to support the simulation of the inspections of the validations cases defined.
Another major objective of PICASSO was to focus on Probability of Detection (POD) issues and deal with the introduction of simulated data in POD studies.
To achieve these goals, the first activity was to focus on the uncertainties modelling strategy. The influent NDT parameters whose variations have a strong impact on the result were defined and the statistical laws for these parameters variation (Design of numerical experiment – DoE) were determined. The DoE tools have been prototyped and the corresponding software modules are available.
The second activity was to work on the combination of experimental and modelling data and to develop a methodology for the transfer function POD approach. A transfer function approach has been proposed to obtain POD curve in Alumium based on simulation data in Titanium and Aluminium. Results were evaluated on a ET case provided by EADS (HFET on fatigue cracks). This new approach shows how linear methods can be extended to account for piecewise linear relationships between signal and crack length. The new methodology requires linear piecewise signal fitting. An heuristic method has been proposed to compute a piecewise/mixture POD. The proposed methodology to generalize the transfer function approach to piecewise linear relationships between signal and crack length requires the following steps :
• Step 1: to estimate parameters of the piecewise linear regressions for simulated data for Titanium and Aluminum and for experimental data (fatigue cracks) for Titanium,
• Step 2: to collect the three associated breakpoints denoted , , and thus to define the four resulting intervals on which computing the predicted piecewise mean transferred signal for Aluminum,
• Step 3: to build and to apply the piecewise transfer function to each of the intervals obtained at step 2,
• Step 4: to compute a POD curve for the detection of fatigue cracks in Aluminum.
The third activity was the improvement of the POD estimation models. Classical Berens models (parametric) for estimation have been analysed and limits and hypotheses of the models have been highlighted so that to orientate developments on alternative approaches.
Alternative non-parametric approaches have been developed (Quartile regression and kernel regression) to deal with the cases that cannot be dealt with parametric methods.
Furthermore, developments of analysis tools aiming at checking the consistency of the input data towards these hypotheses have therefore been carried out. These tools include :
• Plots of residual graphs (difference between actual data and regression fit);
• Quantitative evaluation of the noise;
• Alternative data representation for evaluation of the regression fit.
The prototype implementation of specified methods have been made.
Finally, the project’s library in C++ useable by all the NDT software of the project have been developed.
The assessment of the PICASSO new POD methodology has been made using the validations cases. These validation cases have been manufactured. For these validation cases several NDT laboratories have carried out the experimental NDT investigations. Furthermore simulations of these experiments have been done. Based on these experimental and simulated data the POD curves have been calculated. The results have been compared and the equivalency have been described. For some validation cases, both POD calculations agreed very well. For every NDT method investigated, ET, RT and UT, at least one validation case could be successfully carried out. For every method the measured and simulated data agreed very well and it could be shown that simulation of NDT methods could be used to calculate POD curves with sufficient confidence.
• Evaluation of the POD calculation methodology for ET POD calculation
Simulation-based POD curves have been determined and compared to experimental POD curves for an application case consisting of High Frequency Eddy Currents Testing of fatigue cracks in flat Titanium parts. The simulation results have been obtained using the POD module proposed in the CIVA software. Simulation-based POD results are in good agreement with experimental POD for this configuration. It is shown that by taking into account a limited number of variability sources (start scan position, crack high, probe angle and electrical contacts on cracks), a simulation-based POD curve with features very similar to the experimental curve is obtained. From this successful attempt, the basic methodology and principle is considered with confidence.
An approach of confidence evaluation for POD using Uncertainty Propagation through simulation codes has been proposed and demonstrated on two different NDT application cases. The so provided confidence has a different meaning than the classical confidence estimation based on the sample size but has the same objective - integrate quantitatively the level of available knowledge in the process description – and results in a robust process to determine POD curves using uncertainty propagation through simulation.
• Evaluation of the POD calculation methodology for RT POD calculation
With the help of wedges and sheet metal with drilled holes, the POD of pores in a microradiograph has been measured experimentally. The whole configuration has been simulated successfully using aRTist including the image evaluation by the human operator. For small indications with good experimental support, an excellent agreement of the POD within a few percent has been found. For larger indications, good qualitative agreement is observed. Adjustments have been done to the simulation software to closely match the experimental data, which readily lead to a simulation supported POD methodology.
The proposed simulation supported POD methodology for RT has been successfully tested.
• Evaluation of the POD calculation methodology for UT POD calculation
The recommended general method for performing a simulated supported UT POD study was developed.
Using a combination of empirical and simulated POD data has several advantages, if only to improve the confidence that simulated approach has correctly replicated the inspection scenario. The number of real defects will depend on the requirements and reason choosing a combined approach.
The main objective of the project is fulfilled.
A recommendation for simulation based POD calculation was derived and a final public Workshop has been held.
The concept of “simulation supported POD” has thus been validated on some first realistic results.
Potential Impact:
To cover this question, a questionnaire was sent and filled by the partners (milestone M6.2). A summary of the answers is proposed here-under :
1. Which are the target “customers” of the concept of Simulation Supported POD developed in the framework of PICASSO ?
Any company, from any industrial sector, involved in concerns about NDE methods reliability with reduced costs, to ensure appropriate answers to customers (request for justification of proposed procedures) or to their own needs (new methods, modification of existing methods due to evolution of requirements or adaptation to materials and systems – eg removal of existing probes or devices…)
2. List the main Strengths of the concept of Simulation Supported POD developed in the framework of PICASSO ?
The simulated POD approach developed by PICASSO enables more reliable and low cost POD studies to be carried out. It will also enable quantitative reliability assessments of NDT techniques when an empirical study is not practical or not possible.
Increase knowledge of NDE procedures, limitations and influencing factors and of the underlying physics This understanding is likely to lead to improved inspection reliability.
It enable also fast technique capability estimations and a decrease the need for experimental test pieces
3. List the main Weaknesses of the concept of Simulation Supported POD developed in the framework of PICASSO ?
Difficulties arise in determining all input data relevant for simulation and full exploitation of acquired data.
It is not always easy to achieve the fact that :
• A validated model is required.
• A thorough understanding of factors that influence the inspection is required.
It could be hard to justify the use of only simulated data in order to estimate NDE procedures because of the difficulties to model the real complex flaw geometries and some procedure parameters may be difficult to model and must be justified experimentally.
4. Which elements of the PICASSO results / methodologies need strengthening in order to become more exploitable ?
Currently it is not exploitable for all inspections. Through the PICASSO project the capability of NDT models has increased. The simulated POD approach could be used for more inspections if NDT models were further improved. Also, a recognized method for validating NDT models would be useful.
A more comprehensive validation of the different models (in UT,ET,RT) would be necessary prior to their use for simulation supported POD.
5. What obstacles or difficulties might you face when trying to exploit the PICASSO results / methodologies ?
From a technical point of view, determination of input data related to uncertainties sources and their related distributions and margins remain difficult to establish and may cause discrepancies between POD based on simulated data and POD based on experimental data, as well as bias over mixed POD data. Other potential difficulties arise when combining several codes and softwares (exploitation of data through different formats, etc…) .
The main issue is the understanding of factors that influence the inspection reliability. The PICASSO project developed methods to carry out a study to do this. However, understanding how well these factors are understood can be challenging. Members of the PICASSO group are continuing to work on this issue.
6. Please list all the opportunities and plans that your organization has for the exploitation of the PICASSO results / methodologies
PICASSO has increased the quality and reliability of POD estimations, simplified planning of POD studies, increased NDE knowledge and understanding of POD and increase the use of simulation tools for inspection of engine components.
Academic partners :
• further development of RT simulation tool
• use/distribution of POD tools
• supervision of doctoral students
• presentation of results at 5th European-American Workshop on Reliability of NDE, Berlin, 2013
• exploitation of the developments within the next version (commercial or not) of the simulation softwares which are to be released soon.
Industrial partners :
• incorporation of the methods developed in PICASSO in the industries’s procedures to determining the reliability of its inspections.
• ability to carry out simulated POD studies is a key part of the industries NDT strategy.
• support for the damage tolerance design
• evaluation of the absolute performance of ET, UT and RT method in the framework of critical parts inspection
• presentation of results at 5th European-American Workshop on Reliability of NDE, Berlin, 2013
7. Please describe what could be the plan to bring the PICASSO technology to higher TRL (6-7)
The techniques developed by PICASSO require further use to gain improved knowledge of how to implement simulated POD methods.
Members of the PICASSO group are currently working on a complementary document describing the recommended methods for carrying out simulated POD studies. The aim will be to capture the knowledge gained during the PICASSO project and have an agreed method that can be tested by the industrial partners. It is hoped that this will lead to common accepted methods.
The importance of the work can be measured by the fact that the PICASSO group members are continuing to work (after the project is finished) towards a common method that will allow the knowledge gained through the project to be used by industry.
Furthermore, it is already planned to continue and reinforce collaborations with other groups such as the MAPOD group in USA for an improved leverage about the overall use of simulation and dedicated tools (such as simulation supported POD) in industrial sectors. Such a collaboration with the MAPOD group has already been intiated by the PICASSO partners. Also, needs for better acceptance of simulation by standardization /certification organizations. To address this point, it is also planned to continue the communication with the RISK Group : PICASSO results will be presented during their next European meeting in June 2013.
List of Websites:
http://www.picasso-ndt.eu
Relevant contact details :
Snecma (Coordinator) : MALEO Nancy - +33 (0)1 60 59 91 12 - nancy.maleo@snecma.fr
BAM (Point of Contact for aRTist software) : BELLON Carsten - + 49 - 30 - 8104 3658 - carsten.bellon@bam.de
CEA (Point o Contact for CIVA software) : MAHAUT Steeve - +33 (0)1 69 08 47 14 - steve.mahaut@cea.fr
Chalmers University (Point of contact) : WIRDELIUS Hakan - +46(0)31772 27 12 - hakan.wirdelius@chalmers.se
EADS IW (Point of contact) : DOMINGUEZ Nicolas - 33 (0)5 61 16 88 13 - Nicolas.Dominguez@eads.net
HTS (Point of contact for VIC3D software) : NADLER Michael - + 49 (0) 3523 77560 - michael.nadler@htsdd.de
IZFP-D (Point of contact for EFIT Software) : SCHUBERT Franck - +49 (0) 351 / 88815-523 - Frank.Schubert@izfp-d.fraunhofer.de
MTU (Point of contact) : BARON Hans-Uwe - +49 (0)89 14 89-26 46 - hans-uwe.Baron@muc.mtu.de
PHIMECA (Point of contact for the new POD tools) - YALAMAS Thierry - +33 1 58 51 18 02 - yalamas@phimeca.com
Rolls-Royce plc (Point of contact) : BARDEN Tim - +44 (0) 117 9797143 - Tim.Barden@Rolls-Royce.com
TURBOMECA (Point of contact) : JULIAC Etienne - +33 (0)5 59 12 10 47 - Etienne.juliac@turbomeca.fr
TWI (Point of contact) : WALLACE Philip - +44 (0)1639 873100 - philip.wallace@twi.co.uk
GKN (Point of contact for GKN software) : ROSELL Anders - +46 (0)520 94080 - anders.ar.rosell@gknaerospace.com
Technic Control (Point of contact) : LYSZCZYK Pawel - + 48 605 081 489 - p.lyszczyk@technic-control.com.pl