Final Report Summary - ECO2PAINTER (Economical and Ecological High Quality Painting at Highly Scalable Batch Sizes)
European industry is in a transition from mass production industry towards a more competitive, knowledge-based, customer- and service-oriented one. Production on demand, mass customisation, rapid reaction to market changes and quick time-to-market of new products and variants at small batches are needed - at low cost and high quality. Highly flexible, scalable and user-friendly production equipment is needed, including robotic systems for painting a common process in production.
The necessity for the project rises from significant disadvantages of current automatic painting systems, robotised as well as non-robotised ones. Within this project a new technology was developed which enables SMEs to customise efficiently and to rapidly and smoothly launch new products. ECO2PAINTER aimed at the development of a novel technology for self-programming painting robots that are capable of learning. Project results allow eco-efficient high-quality painting even of very small lot-sizes with downtimes during product changes that are 10-100 times shorter than those of conventional systems. Within this project, a system was developed that uses sensors to reconstruct, recognise and decompose the products to be painted. Using this information, the process was automatically planned and robot programs were generated and executed subsequently. Quality improvements were achieved automatically or by ergonomic user interaction and were re-applied in planning future product variants. The resulting improvements in the competitiveness will improve the return of investment (ROI) in eco-friendly painting lines.
Strategic objectives:
- Development of a prototype for a robotic painting-system that programs itself by use of active 3D-vision and a "what we see is what we paint" approach at high quality.
- Automatic "closed loop" application programming within minutes instead of hours / days / weeks for minimal downtimes at product changes and mixed production.
- Transferring knowledge and optimisations from one variant to the next.
- Reduction of used paint and increased quality.
- Installation and field tests with demonstrator and preparation of exploitation of results.
Project objectives:
- Novel robotic system for flexible high-quality painting of highly scalable batch sizes. Down-time due to product-changes will be in the range of seconds / minutes, in comparison to downtimes or delays of conventional systems which are around hours / days / weeks.
- Retrofit a demonstrator into the robotic painting line of end-user FKI's to perform real world evaluation and benchmarking against existing technology.
- Novel or radically improved system components and tools:
a) active three-dimensional (3D) vision comprising reconstruction, recognition, localisation, decomposition;
b) automatic paint process planner and motion planner for complex surface geometries;
c) scheduler for creating near optimal multi-robot schedules (criteria: manual teach in);
d) for simulating the full application unveiling critical regions;
e) for interactive and automatic (local) adaptation / optimisation of paint strokes.
Knowledge and methodologies on dealing with uncertainty and variance:
- through 3D localisation, recognition and decomposition of complex 3D objects even for very large part variances;
- by novel (investigative) control of active range sensing, driven to retrieve missing information (geometrical uncertainties) and to reduce ambiguities of hypotheses.
Knowledge, methods and tools on dealing with quality and efficiency in automatic planning:
- through embracing automatic process / motion planning and simulation with adaptation / optimisation in a closed loop system and investigate convergence towards the criteria (quality, time, paint-usage) by iterative optimisation and learning functionality;
- through improved scheduling swapping tasks from one robot to another.
User interaction dealing with workers and knowledge:
- by incorporating the worker's expertise and experience for quality improvements without production downtimes by user interaction-tools using.
ECO2PAINTER enhances a "what you see is what you paint" approach. Efficiency is reached by automatic planning and flexibility is reached by sensing. Even completely unknown parts to be painted are (1) scanned while transported by the conveyor. (2) Next, the sensor information is interpreted and (3) used by planners to automatically plan paint strokes and robot motions. Next, the entire application is simulated (4) and the predicted painting result is used to control (5) either additional quality improvements or to execute the automatically generated programs on the robots (6). Quality improvements are done by (a) automatically adapting / optimising the paint strokes, or (b) by the operator interacting via a simple augmented graphical animation of the results and the planned strokes. Learning capabilities (supervised) allow a transfer of optimisations to next variants.
The major innovation beyond state-of-the-art is clearly the novel system that efficiently paints products even at very small batch sizes at very high quality with down-times near zero. Innovative features of ECO2PAINTER are:
1. Robust active 3D sensing and recognition for large sets of parts and complex shapes.
2. "Closed loop" process and motion planning and scheduling.
3. Efficient interfaces for non-robotic experts.
A SW tool for accurate 3D position / orientation determination, based on algorithms provided as background has been developed. The prototype tool is able to localise complex objects / parts robustly even if strong occlusion (e.g. by overlapping parts) appears. PRO developed mathematical methods to recognise complex shaped 3D objects in an industrial setting. These implemented methods were tested in the laboratory as well as in the production plants of the SME end user where there is much of clutter, occlusions and background noise. The system demonstrates robust active scanning with high robustness against part motion and poor surface conditions. A simple user interface allows setting up the system in an intuitive way.
In order to enable a closed loop process a variety of distinguishable methods have been developed, implanted and integrated in a common framework.
- Automatic paint process planner and motion planner for complex surface geometries featuring:
a) possibility to use flexible stroke types on non-flat surfaces;
b) advanced methods for specification of path smoothness and process angles for paint tool;
c) light process simulation to discover unnecessary strokes and overspray;
d) possibility to specify defined painting directions for decomposed parts with known normal directions;
e) possibility to force directions of paint motions according to cell specific painting directions;
f) possibility to force directions of paint strokes according to part directions;
g) enabling painting of slightly curved surfaces using control points in paint strokes directly in the automatic process planning;
h) API allowing close integration and advanced communication between paint planning-, scheduling- and path planning software;
i) tasks can be specified with "Soft task constraints" allowing the path planner to deviate from the specified (optimal) path.
- A novel scheduler has been developed which has been closely integrated with the motion planner.
- A paint simulator for unveiling critical regions.
- An optimisation / adaptation procedure to optimise generated paint strokes.
An efficient interface for non-robotic experts has been developed featuring:
- improved InropaBasic allowing flexible editing of paint strokes;
- 3D-virtual reality technology developed at AAU has been integrated with the Inropa basic program. The resulting system (RobTeach) removes the need for robot know-how in the programming phase.
The necessity for the project rises from significant disadvantages of current automatic painting systems, robotised as well as non-robotised ones. Within this project a new technology was developed which enables SMEs to customise efficiently and to rapidly and smoothly launch new products. ECO2PAINTER aimed at the development of a novel technology for self-programming painting robots that are capable of learning. Project results allow eco-efficient high-quality painting even of very small lot-sizes with downtimes during product changes that are 10-100 times shorter than those of conventional systems. Within this project, a system was developed that uses sensors to reconstruct, recognise and decompose the products to be painted. Using this information, the process was automatically planned and robot programs were generated and executed subsequently. Quality improvements were achieved automatically or by ergonomic user interaction and were re-applied in planning future product variants. The resulting improvements in the competitiveness will improve the return of investment (ROI) in eco-friendly painting lines.
Strategic objectives:
- Development of a prototype for a robotic painting-system that programs itself by use of active 3D-vision and a "what we see is what we paint" approach at high quality.
- Automatic "closed loop" application programming within minutes instead of hours / days / weeks for minimal downtimes at product changes and mixed production.
- Transferring knowledge and optimisations from one variant to the next.
- Reduction of used paint and increased quality.
- Installation and field tests with demonstrator and preparation of exploitation of results.
Project objectives:
- Novel robotic system for flexible high-quality painting of highly scalable batch sizes. Down-time due to product-changes will be in the range of seconds / minutes, in comparison to downtimes or delays of conventional systems which are around hours / days / weeks.
- Retrofit a demonstrator into the robotic painting line of end-user FKI's to perform real world evaluation and benchmarking against existing technology.
- Novel or radically improved system components and tools:
a) active three-dimensional (3D) vision comprising reconstruction, recognition, localisation, decomposition;
b) automatic paint process planner and motion planner for complex surface geometries;
c) scheduler for creating near optimal multi-robot schedules (criteria: manual teach in);
d) for simulating the full application unveiling critical regions;
e) for interactive and automatic (local) adaptation / optimisation of paint strokes.
Knowledge and methodologies on dealing with uncertainty and variance:
- through 3D localisation, recognition and decomposition of complex 3D objects even for very large part variances;
- by novel (investigative) control of active range sensing, driven to retrieve missing information (geometrical uncertainties) and to reduce ambiguities of hypotheses.
Knowledge, methods and tools on dealing with quality and efficiency in automatic planning:
- through embracing automatic process / motion planning and simulation with adaptation / optimisation in a closed loop system and investigate convergence towards the criteria (quality, time, paint-usage) by iterative optimisation and learning functionality;
- through improved scheduling swapping tasks from one robot to another.
User interaction dealing with workers and knowledge:
- by incorporating the worker's expertise and experience for quality improvements without production downtimes by user interaction-tools using.
ECO2PAINTER enhances a "what you see is what you paint" approach. Efficiency is reached by automatic planning and flexibility is reached by sensing. Even completely unknown parts to be painted are (1) scanned while transported by the conveyor. (2) Next, the sensor information is interpreted and (3) used by planners to automatically plan paint strokes and robot motions. Next, the entire application is simulated (4) and the predicted painting result is used to control (5) either additional quality improvements or to execute the automatically generated programs on the robots (6). Quality improvements are done by (a) automatically adapting / optimising the paint strokes, or (b) by the operator interacting via a simple augmented graphical animation of the results and the planned strokes. Learning capabilities (supervised) allow a transfer of optimisations to next variants.
The major innovation beyond state-of-the-art is clearly the novel system that efficiently paints products even at very small batch sizes at very high quality with down-times near zero. Innovative features of ECO2PAINTER are:
1. Robust active 3D sensing and recognition for large sets of parts and complex shapes.
2. "Closed loop" process and motion planning and scheduling.
3. Efficient interfaces for non-robotic experts.
A SW tool for accurate 3D position / orientation determination, based on algorithms provided as background has been developed. The prototype tool is able to localise complex objects / parts robustly even if strong occlusion (e.g. by overlapping parts) appears. PRO developed mathematical methods to recognise complex shaped 3D objects in an industrial setting. These implemented methods were tested in the laboratory as well as in the production plants of the SME end user where there is much of clutter, occlusions and background noise. The system demonstrates robust active scanning with high robustness against part motion and poor surface conditions. A simple user interface allows setting up the system in an intuitive way.
In order to enable a closed loop process a variety of distinguishable methods have been developed, implanted and integrated in a common framework.
- Automatic paint process planner and motion planner for complex surface geometries featuring:
a) possibility to use flexible stroke types on non-flat surfaces;
b) advanced methods for specification of path smoothness and process angles for paint tool;
c) light process simulation to discover unnecessary strokes and overspray;
d) possibility to specify defined painting directions for decomposed parts with known normal directions;
e) possibility to force directions of paint motions according to cell specific painting directions;
f) possibility to force directions of paint strokes according to part directions;
g) enabling painting of slightly curved surfaces using control points in paint strokes directly in the automatic process planning;
h) API allowing close integration and advanced communication between paint planning-, scheduling- and path planning software;
i) tasks can be specified with "Soft task constraints" allowing the path planner to deviate from the specified (optimal) path.
- A novel scheduler has been developed which has been closely integrated with the motion planner.
- A paint simulator for unveiling critical regions.
- An optimisation / adaptation procedure to optimise generated paint strokes.
An efficient interface for non-robotic experts has been developed featuring:
- improved InropaBasic allowing flexible editing of paint strokes;
- 3D-virtual reality technology developed at AAU has been integrated with the Inropa basic program. The resulting system (RobTeach) removes the need for robot know-how in the programming phase.
