Skip to main content
European Commission logo
Deutsch Deutsch
CORDIS - Forschungsergebnisse der EU
CORDIS
CORDIS Web 30th anniversary CORDIS Web 30th anniversary
Inhalt archiviert am 2024-06-18

Intervention Research On Health Literacy among Ageing population

Final Report Summary - IROHLA (Intervention Research On Health Literacy among Ageing population)

Executive Summary:
Why is health literacy important for EU member states?
Health literacy is the degree to which people are able to access, understand, appraise and communicate information in relation to health and disease. In Europe only half of the adult population has sufficient levels of competencies linked to health literacy. In particular, older persons, people with a lower level of education, with lower socio-economic status or from migrant or minority communities face health literacy problems.

Research in the IROHLA project
The project Intervention Research on Health Literacy of the Ageing Population in Europe (IROHLA) focused on producing a set of feasible interventions that would improve health literacy outcomes.
Research in the IROHLA project showed that better health literacy outcomes could be achieved when interventions take place in four areas:
• Empowerment of the older persons with low health literacy
• Strengthening the social support systems: family, caregivers, communities
• Enhancing the communication and interaction competencies of health workers
• Improving the health system, to become more accessible for all groups in society
The IROHLA project brought together 20 shining examples of health literacy interventions in these four areas from around the world and validated interventions for the European context.
The IROHLA project concluded that incorporating health literacy in all policies in the area of health care or healthy ageing, is likely to lead to more effective programmes. For example, public health programmes, as well as patient safety and health care quality programmes benefit from easily understandable health information for the population and more accessible and acceptable e-health and m-health applications. Families, communities and volunteers can effectively assist older people in healthy living. Health organisations can become more health literacy friendly.

Guidelines for policy and practice
The project looked into necessary steps for implementing health literacy in government policies:
1. National and regional governments set the standards for accessibility to health promotion, prevention, cure and care. Focus on health literacy will contribute to achieving equity and sustainability.
2. Ministries of health can use the IROHLA project results to produce a national health literacy policy or strategy, as several countries in the EU already have done, guiding the health and welfare sector. A comprehensive health literacy approach is effective, focusing not only on individuals and communities, but also on health professionals and organisations.
3. In health services person-centred care has proven to be beneficial: when health care workers are able to communicate effectively with older people, adherence to medical treatments improves. When hospitals and health facilities become more accessible, older people can more easily find their way, make appointments, or understand written or oral information.

What is the expected impact?
Improving the health literacy of older people will improve their capacities to stay healthy and manage chronic conditions. It will give them access to innovative communication technologies. It will enhance adherence to medical treatments. It will result in more effective and sustainable health systems, due to more efficient utilisation of health services. It will increase equity in access to health services. It will make healthcare more efficient. Most of all, it will contribute to active and healthy ageing and increasing healthy life expectancy, one of the EU targets for 2020.

Project Context and Objectives:
2.1 Context

The Intervention Research On Health Literacy of the Ageing population in Europe (IROHLA) was a European project funded by the 7th Framework Programme (Grant 305831). The project started in December 2012 and lasted three years, until 30 November 2015.
Health Literacy
Health literacy has been defined as the degree to which people are able to access, understand, appraise and communicate information to engage with the demands of different health contexts in order to promote and maintain good health across the life course. Health literacy therefore is an essential asset to stay healthy and to recover quickly when falling ill. Low health literacy is associated with lower life expectancy, poorer health status, and more chronic illnesses. People with low health literacy make more use of health services than people with higher levels of health literacy. People with low levels of health literacy often have lower levels of education, and belong to lower social economic groups in the society.
The European Union has identified health literacy as one of the areas where improvement is needed, in order to improve equity in health, improve the health status of the population, and increase the effectiveness of healthcare services. A survey into health literacy in Europe in 2010 showed that 30% to 50% of the population experiences limitations in health literacy and that ageing people are faced more with this problem than younger people, due to compounded mental, social and economic conditions. The European Innovation Partnership of Active and Health Ageing has embraced health literacy for the ageing population as one of the priorities for healthy ageing. In the past years most projects were mapping the problems and identifying the vulnerable groups with low health literacy; this project has built a body of knowledge on best ways to tackle health literacy issues of the ageing population.
The IROHLA project Consortium
The IROHLA consortium was lead by the University Medical Center Groningen (UMCG) and consisted of 22 partners: academic institutions, health promotion organisations, network organisations for health promotion and healthy ageing, health insurance companies, as well as business companies operating in the health sector. The consortium covered nine countries, but because of incorporated network organisations it could reach nearly all EU member states (see table 1 below). The broad interdisciplinary composition of the consortium brought together knowledge from science, practice and interest groups. The inputs of business companies mainly in the domain of Information and Communication Technology helped to produce innovations. Therefore the focus of the project is very much on feasible interventions, which can be applied further after completion of the project.

Table 1 Partners in the IROHLA consortium

Name Institution Country
1 University Medical Center Groningen (UMCG) The Netherlands
2 CBO The Netherlands
3 University of Groningen (RUG) The Netherlands
4 Jacobs University, Bremen Germany
5 Baltic Region Healthy Cities Association Finland
6 National University of Ireland, Galway Ireland
7 Norwich Medical School, Faculty of Medicine & Health Sciences England
8 National Institute for Health Development Hungary
9 EuroHealthNet Belgium
10 Institute of Preventive Medicine, Environmental & Occupational Health - Prolepsis Greece
11 Italian National Institute on Aging (INRCA) Italy
12 German Institute for Health Promotion (BZgA) Germany
13 AGE – Platform Europe Belgium
14 European Social Insurance Platform (ESIP) Belgium
15 Regional Agency for Health Marche Region Italy
16 Hanze University of Applied Science Groningen The Netherlands
17 Cambo Industries Digital Greece
18 Live Online Coaching Germany
19. IP-Health Vitalinq The Netherlands
20. Educational TV-NL (ETV) The Netherlands
21. Noordhoff Publishers The Netherlands
22. Federal Association of Health Insurances in Germany (AOK) Germany

Partner no. 2 CBO was terminated as member of the consortium due to dissolving the organisation after bankruptcy on 7 September 2015.
2.2 Project objectives and activities

Objectives
The main objective of the IROHLA project was to introduce in member states of the European Union evidence-based guidelines for policy and practice for a comprehensive approach improving health literacy of the ageing population.
The project aimed to:
(i) Contribute to the understanding of health literacy in different European contexts and develop a comprehensive model for addressing health literacy needs in older adults in various settings (WP2)
This encompasses:
a. An analysis of empowering interventions addressing determinants of health literacy in the ageing population at individual, community and society level and
b. An analysis of health literacy communication interventions to mitigate the effects of low health literacy levels in health service delivery,
(ii) Develop a manual for assessment of quality and feasibility of health literacy interventions/activities in ageing population (WP3)
(iii) Assess the quality and feasibility of interventions or practices in the ageing population, which will contribute to improvement of health literacy:
a. In the health care sector: assess good practices addressing the needs of the target group (both empowering and health literacy communication in practice and in training). (WP4)
b. In the commercial sector: assess good practices which aim at empowerment and enhanced communication with ageing population people (WP5)
c. In the social sector: good practices for improved accessibility to and utilisation of services by older people with low literacy levels (WP6)
(iv) Select from the long list of interventions a set of maximum 20 interventions which together constitute a comprehensive approach of addressing health literacy needs of the ageing population (WP7)
(v) Develop and evidence-based guideline for policy and practice for EU member states and regions:
a. Validate and if necessary adjust selected evidence-based interventions, aiming to improve the capacity of the ageing population in obtaining, processing and understanding of information in the health care sector, as well as health literacy communication interventions in the health sector.
b. Present a guideline for policy and practice at national, regional and local level, which shows policymakers the specific and important measures they can take, and implementers practical interventions to initiate. (WP8)
(vi) Actively disseminate the results through media and information channels, in collaboration with partners, to create awareness amongst stakeholders and to promote adoption and implementation of guidelines for policy and practice. The project wanted to provide information to relevant stakeholders in the area of healthy ageing to integrate the research findings in European programmes in various contexts. (WP9)

Activities in the project
The IROHLA project had nine Work Packages (WPs), of which the first was management and the ninth was communication and dissemination. In the project three phases could be distinguished:
1. Defining general principles and a model for analysing health literacy interventions for the ageing population as well as a theoretical framework for the classification of interventions. This enabled the project to better understand the components and effective mechanisms of health literacy interventions (in WP2 during the first 9 months of the project);
2. Making an inventory in the international health, social and commercial sectors of health literacy and other literacy interventions and effective mechanisms. Using the theoretical framework viable components were identified, which can be applied in the European health sector (in WP3 to WP7 during the second 10 months of the project);
3. Validating a comprehensive set of at least 20 feasible interventions and defining guidelines for policy and practice for improving health literacy of the ageing population in European member states (in WP8 during the remaining 17 months of the project).

WP2 produced the theoretical framework and used a mixed-methods approach directed to multiple domains, determinants, interventions and outcomes of health literacy. Researchers conducted a series of systematic reviews and assessments to evaluate the available evidence on health literacy in three areas: adherence to medical plans, on the comprehensibility of health related documents, and on the influence of the social, cultural and physical environment on the effects of health literacy interventions, all focusing on older adults.
Researchers performed a quantitative study into health literacy and compliance to guidelines for physical activity and nutrition guidelines, and into the mediating role of behavioural determinants, and into Health literacy and self-management among adults aged 75 and over and the influence of socio-demographic background. Researchers conducted a series of participatory focus group discussions around the needs and perceptions of older adults to understand the perspectives of older adults and health professionals. Furthermore, researchers consulted stakeholders in brainstorming sessions and in a multi-phase Delphi study.
WP3 produced a manual with instructions for a search strategy of interventions in the area of health literacy, communication and social inclusion. The theoretical framework developed in WP2 was translated into a set of practical instructions to search in scientific databases, grey literature, information websites and other sources of information. The manual also provided instructions for the selection process and for inclusion criteria as well as formats for description of identified interventions.
WP4 performed a search into relevant interventions in the health sector. The initial number of 6556 scientific publications and 1382 other publications was reduced in a selection process to 233 relevant interventions, which were grouped according to the focus, e.g. a disease or health condition, or according to the target group. Finally the interventions were scored using a set of agreed criteria. The 20 highest scoring interventions were described in detail.
WP5 went to a similar process of identifying relevant interventions in the commercial sector, where for-profit players or non-traditional players were involved in activities in health or social inclusion of older adults. Out of 3496 scientific publications and 1024 other sources of information (mainly internet), 93 were scored using defined criteria and the 20 highest ranking were described in detail. In some cases clusters of interventions were described as they had many similarities.
WP6 did the same for the social sector, but also performed an online survey among experts in health literacy and health communication. In total 3576 scientific publications were identified, 199 publications mainly on internet and 46 interventions which were suggested by experts in the survey. After the selection process 185 interventions remained which were scored. Clustering of interventions with similar components was done, to capture the broad experience in the field. Again the 20 best interventions were selected.
WP7 provided further analysis of the quality of the sixty interventions forwarded by the WPs 4, 5 and 6. With a systematic checklist selection was made, resulting in 20 most relevant interventions that should be considered for a comprehensive approach, addressing health literacy of older people. In addition, components of interventions identified in the previous work packages, were categorised and the working mechanisms behind these interventions were further analysed, based on the taxonomy (developed in WP2) and the target groups.
WP8 concentrated on the validation of interventions. Validation in this context means building further on existing evidence, using effective intervention strategies identified in earlier literature research and trying to implement activities in different country settings. The following working groups operated under WP8:
• WG1: Community and communication, studied in one cluster the role of communities in strengthening health literacy by interacting with the health sector professionals, and studied in a second cluster innovation means of communication with older people (photo novelas). Studies took place in the Netherlands, Italy, Hungary and Germany
• WG2: E-health and M-health, tested a variety of interventions using ICT, e.g. in health promotion, health education, home based care, virtual communities. The common theme of the studies was to identify criteria making those applications better accessible for older people with low health literacy. Studies took place in Germany, the Netherlands, Greece and Italy.
• WG3: Tested modules for capacity building of professionals in improving patient-centred communication, using education approaches in various settings of health service delivery in different countries, identifying the main factors for successful capacity building. Studies took place in the Netherlands, Ireland and Italy.
• WG4: Tested scans for health literacy friendly environments in health institutions, assessing the effectiveness of instruments to measure accessibility and barriers to health services in different setting and countries, identifying effective functionalities of tools. Studies took place in the Netherlands, Ireland and Finland.

In WP8 the Consortium partners produced guidelines for policy and practice:
• Overall policy brief for governments,
• Policy brief for health organisations with guidance for development of health literacy interventions
• Specific guidelines, recommendations and best practices for implementers of health literacy interventions with references to existing guidelines and background literature.

Project Results:
3 Main Results

3.1 Conceptual Framework for Heath Literacy Interventions

The initial phase of the project concentrated on producing a conceptual framework for health literacy interventions. This would enable the classification of actions and the identification of evidence-based components suitable for a comprehensive health literacy programme.
The work was divided in systematic reviews, taking stock of the state of the art in relevant areas, qualitative and quantitative research to understand the viewpoints of older people, and production of an intervention model.

3.1.1 Systematic reviews

The project performed three systematic reviews in the initial stages.

Meta Review of Health Literacy and Adherence
Findings:
• Adherence interventions covered a wide range of strategies with several of these being effective in improving adherence, independent of level of health literacy.
• Strategies like additional guidance via telephone counselling and diabetes educational classes were promising; these might even decrease health disparities due to low health literacy. Their effectiveness required further study, as evidence was limited.
• Evidence on the association between health literacy and adherence in older adults was inconsistent. Longitudinal studies should be undertaken to understand this relationship and to assess which intervention strategies could be worthwhile.
Literature Review of Health Literacy and Comprehensibility of Health Related Documents
Findings:
• Health related documents that include narrative formats and apply multimedia, or apply multiple-feature revisions, were promising. They reduced comprehension problems in older adults with low health literacy.
• Evidence was inconsistent about the effectiveness of many other interventions aiming to improve the comprehensibility of health related documents in older adults with low literacy.
• A promising route to obtain evidence on the processing of health-related documents in older adults was identified assessing cognitive and comprehension processes. Testing of health related documents in the target population was pivotal to guarantee comprehensibility of these documents for older adults with low health literacy.

Literature Review of Influence of the Social, Cultural and Physical Context on the Effects of Health Literacy Interventions
Findings:
• The review resulted a classification of contextual factors and summarised the evidence regarding the influence of social, cultural and physical factors on (the effects of interventions to improve) health literacy outcomes as described in the available reviews.
• There was some evidence that the negative consequences of limited health literacy might be mitigated by social support and characteristics of the health system.
• We found no studies that examined differences in intervention effects by contextual factors.
• The influence of context on the effectiveness of health literacy interventions deserved further study, in particular with regard to social support and characteristics of the healthcare system.

3.1.2 Understanding health literacy: research in older adults

Health literacy and compliance to guidelines for physical activity and nutrition guidelines and the mediating role of behavioural determinants – first quantitative study

Findings:
• Health literacy was significantly associated with compliance to guidelines for sufficient physical activity but not with compliance to guidelines for fruit and vegetable use, among older (55+) adults.
• Self-efficacy mediated the association between health literacy and compliance to guidelines for sufficient physical activity. Attitude and risk perception did not have no clear mediating role in the pathway from health literacy to compliance to physical activity guidelines.
• Improvement of self-efficacy of older adults with low health literacy might be an effective intervention to increase their physical activity.
Health literacy and self-management among adults aged 75 and over and the influence of socio-demographic background - second quantitative study

Findings:
• Low health literacy was associated with poorer self-management abilities independent from socio-demographic characteristics like age, gender, marital status, and income. Higher educated older adults with low health literacy did have a relatively low level of self-management, being the only case of modification of associations by health literacy that we found.
• These results showed that older adults above 75 with low health literacy generally were a vulnerable group for low self-management abilities.
• Early recognition of low health literacy and subsequent interventions to improve health literacy might help to maintain or improve the health of adults aged 75 and over.

Needs and perceptions of older adults: qualitative study on the perspectives of older adults and health professionals. For this qualitative study through interviews and focus group discussion approval was obtained from the Ethical Committee of the University of Groningen.

Findings:
• Older adults in disadvantaged urban neighbourhoods and health professionals involved in prevention or care in four European countries reported the importance for them of having autonomy, a supportive social network, assertive behaviour, trust in the health professional and health system, sufficient income and time with the health professional, clear roles of the health professional, having trust in the health system, less complexity regarding access to services or information, and access to affordable prevention activities.
• Needs of older adults in their context were also shaped in the interaction with health professionals and the health system, which demonstrates a need for taking into account multiple factors from both domains in developing health literacy interventions.
• Although health literacy interventions might have the same objectives in various countries adaptation was needed of the intervention strategies to older adults’ social, cultural and economic circumstances to come to effective interventions.

3.1.3 The development of a health literacy intervention model

The available evidence was integrated in a comprehensive health literacy intervention model that informs the analysis of best practices or interventions. A literature study was undertaken, combined with brainstorming sessions and a Delphi study among experts to help filling the gaps in the available empirical evidence.
• The use of this multifaceted methodology facilitated the development of a parsimonious intervention model and a list of potentially modifiable determinants that are most promising to mitigate the adverse health effects of poor health literacy
• The methodology also yielded a model that supports the collection and categorisation of promising interventions which improve health outcomes in the case of low health literacy

Intervention model
The intervention model (below) shows that in a comprehensive approach for health literacy interventions it is important to focus on the individual and his/her context and the health worker and his/her context. Although health literacy is defined around abilities of individuals, health literacy outcomes are the result of interactions between the stakeholders. Health literacy is a dynamic concept, very much dependent on the context in which it is used. Ageing and chronic diseases demand higher levels of health literacy compared with healthy youth. In the range of interventions there is not only the possibility to empower the individual to become stronger in decision-making on health issues, but also the possibility for the health worker to mitigate the negative effects of low health literacy. Improved health literacy outcomes are not the only factor for achieving healthy ageing, and therefore should be embedded in a larger process of improving health conditions. The social support to older individuals and the health system support to professionals are critical for achieving and maintaining results.

Figure 1 The IROHLA intervention model

The research in WP2 further resulted in:
• A list of constructs, or theoretical intervention mechanisms generally applied in health literacy interventions, aiming at changing behaviour, building capacity, changing organisations. These theories were often based on social psychology, learning theory, systems theory or communication sciences. The obtained constructs enabled the analysis of the theoretical foundation on which interventions are built.
• A list of potentially modifiable determinants of knowledge, skills and attitudes which play a role in improving health literacy of the ageing population, of all stakeholders (the ageing person, the social context, the professional and the health system).
Development and evaluation of a taxonomy of health literacy interventions
WP2 developed a classification system of the objectives and characteristics of health literacy interventions formulated as taxonomy. This enabled the systematic assessment of interventions. The taxonomy was created in the Delphi study and tested in a series of health literacy interventions described in international literature.
• A comprehensive taxonomy was developed for health literacy interventions targeted at (older) adults, professionals or their context.
• The taxonomy has proven to be easy to use, enabling the classification of the contents of a large range of health literacy interventions, and was reliable in classifying the most frequently pursued objectives.
• A substantial number of interventions tested aimed to inform and educate, to teach skills and to enable behavioural changes, but very few interventions aimed at other relevant objectives such as changing the environment.
• The taxonomy was a suitable tool for IROHLA to evaluate the effects of intervention, best practices or innovative approaches, with further improvements to be made in the process of assessing interventions.

Table 2 Taxonomy of health literacy interventions

Objectives Target group Definition
1 To inform and educate Older adults Interventions with the aim to enable (older) adults with limited health literacy to know about and be aware about health protection, health risks or harms and treatment
Professionals Interventions aiming to enable professionals to know about and be aware of health literacy problems, their impact and interventions to tackle health literacy problems
2 To teach skills Older adults Interventions aiming to assist (older) adults in learning or improving skills that help them with a wide range of health literacy related activities such as reading, writing, calculation, finding information, monitoring symptoms, setting goals, communicating with health care professionals
Professionals Interventions aiming to support professionals in learning or improving their skills to reduce health literacy related problems of individuals. Examples of these skills are learning to recognize or assess health literacy problems, skills to inform and gather information from older adults with limited health literacy, using teach-back skills to enable self-management.
3 To support behaviour change and maintenance Older adults Interventions aiming to encourage and motivate older adults to adopt, change and maintain behaviours, such as participation in screening, self-management, adherence to medicines
Professionals Interventions aiming to encourage and motivate professionals to adopt, change and maintain behaviours to decrease or mitigate health literacy problems
4 To strengthen contextual support Older adults Interventions aiming to enhance support by the social network with health literacy related tasks such as reading health documents, interaction with professionals, self-management or health decisions
Professionals Interventions aiming to provide assistance and encouragement to professionals to tackle health literacy problems
5 To facilitate involvement of individuals at the system level Older adults Interventions aiming to enhance involvement of (older) adults in decision making processes on health literacy such as in planning, formulary and policy decisions, research evaluating healthcare services
Professionals Interventions aiming to enhance involvement of professionals in decision making processes on health literacy such as in planning, formulary and policy decisions, research evaluating healthcare services
6 To change the social, cultural or physical environment in order to enhance the effects of health literacy interventions Older adults Interventions aiming to change the social, cultural and physical environment in order to prevent, reduce or mitigate the impact of environmental factors that may decrease the effect of health literacy interventions. For example, giving non-medical support: a social worker that assists individuals with financial issues (e.g. bus ticket to visit a doctor)
Health system/ Professionals Interventions aiming to changing the organization of the health system in order to assist individuals in overcoming non-medical challenges to reach and use the health system or health services (prevention, care and cure). For example, investments to increase the duration of consultations for individuals with limited health literacy, or communication in a quiet environment

Later in the policy brief the taxonomy was presented as three steps for individuals and three steps for organisations:
• Individuals: information, skills development and behavioural change
• Organisations: contextual support, participatory development and implementation, organisational change

3.2 Inventory of Interventions relevant for Health Literacy Policies

The IROHLA project produced an extensive inventory of interventions with relevance for health literacy policies. It did look in the health sector, in the social sector and in the commercial sector. The search went for formal scientific publications, grey publications of reports and evaluations and also internet based activities. The aim was to find feasible interventions, which would contribute to a programme for addressing health literacy issues among the older population.

3.2.1 Health sector interventions

The table below shows the top 20 selected relevant health interventions for the ageing population with low health literacy, as identified in a systematic review.

Table 3 Selected interventions in the health sector
Title of intervention Authors, year Disease or risk factor
1 A randomized trial to improve patient-centred care and hypertension control in underserved primary care patients Cooper et al 2011 Hypertension
2 Effects of self-management support on structure, process, and outcomes among vulnerable patients with diabetes; Seeing in 3-D: Examining the reach of diabetes self-management support strategies in a public health care system Schillinger et al 2008, 2009 Diabetes
3 A patient-centric, provider-assisted diabetes telehealth self-management intervention for urban minorities Carter et al 2011 Diabetes
4 Multisite randomized trial of a single-session versus multisession literacy-sensitive self-care intervention for patients with heart failure DeWalt et al 2012 Cardio Vascular Diseases
5 I'm taking charge of my arthritis": designing a targeted self-management program for frail seniors" Laforest et al 2007 Muscular and Skeleton Disorders
6 Effectiveness of the chronic disease self-management program for persons with a serious mental illness: a translation study
(also Lorig et al 2005 and Farrell et al 2004)
Randomised controlled trial of a lay-led self-management programme for Bangladeshi patients with chronic disease (European implementation of Lorig et al programme) Lorig et al 2014

Griffiths et al 2005 Depression
/Chronic diseases

Chronic disease
7 Developing and testing lay literature about breast cancer screening for African American women; the Delta project: increasing breast cancer screening among rural minority and older women by targeting rural healthcare providers Coleman et al 2003a, 2003b Breast cancer;
8 Activ-ins-Alter, Active Ageing!
Resch & Lang 2008 Non-specific
9 The '10 Keys' to healthy aging: 24-month follow-up results from an innovative community-based prevention program Robare et al 2011 Non-specific
10 Promoting a breast cancer screening clinic for underserved women: A community collaboration Pruthi et al 2010 Breast cancer
11 Disease management to promote blood pressure control among African Americans Brennan et al 2010 Hypertension
12 Impact of a cardiovascular risk control project for South Asians (Khush Dil) on motivation, behaviour, obesity, blood pressure and lipids Mathews et al 2007; Khush Dil CVD, diabetes, exercise, nutrition
13 An advance directive redesigned to meet the literacy level of most adults: A randomized trial Sudore et al 2007 Non-specific
14 Activating community health centre patients in developing question-formulation skills: A qualitative study Lu et al 2011 Communities with HPs
15 Cancer risk communication with low health literacy patients: a continuing medical education program Price-Haywood et al 2010 Communities with HPs
16 Understanding communication of health information: A lesson in health literacy for junior medical and physiotherapy students Doyle et al 2012 Training HPs in HL awareness
17 Laurys Ine Master’s Thesis “Therapietrouw verbeteren bij osteoporose” Ine 2011 Muscular and
Skeleton Disorders osteoporosis
18 Ikäneuvola Ruori (Guidance Centre for older people) NA Non-specific
19 Eksote (Remote monitoring and health coaching in South Karelia) Eksote Chronic, CVD, Diabetes
20 Analysis of the goals achieved and maintained three months after discharge from assistance integrated home care in patients with "homework” Cataldo et al 2010 MSD

Virtually all Interventions aimed to educate, improve patient skills and motivate. Cultural adaptations and actions to change the physical environment (such as paying for travel expenses) were also common. Only six of the final interventions featured the involvement of patients at a systems level (previous research showed this to be the rarest taxonomy objective). Target groups in the final interventions were both older adults and professionals. Professionals were less often the targets of health literacy studies in well-evaluated and documented research, although there have been many programmes to educate professionals about health literacy problems.

3.2.2 Interesting findings from other sectors

In addition to search in the health sector the IROHLA project also performed a literature and internet search in the social and commercial sectors. This provided interesting interventions, which might be applicable in health care settings. The lists of topics below were considered in the final selection of most promising interventions to be included in health literacy programmes.

The following key topics were identified:
- ICT interventions targeting individuals constrained in ADL
- Literacy and life skills training (including financial literacy training)
- Setting and network approaches (for vulnerable groups, e.g. migrants, low SES)
- E-Health, m-Health and online interventions
- Communication strategies
- Learning and sharing knowledge
- Visiting and help service
- In particular, low-threshold services and outreach work are two essential components for best-practices in the social sector.

Innovative elements identified:
- Internet-access through familiar tools (TV) and thus increase social inclusion
- Involvement older adults in community networks where they can learn, e.g. how to use a camera, and thus give them a chance to show where barriers for older people are
- Contact and learning options or services via mobile applications and online interventions
- Volunteer services in informal contexts: social currency (e.g. help in the garden for a bed in the night)

3.2.3 Twenty Most Promising Interventions in a Comprehensive Approach

The quality assessment of over 60 interventions from different sectors and the selection led to the following list of ‘promising’ examples for a comprehensive approach of health literacy for older people. Described are the names and the most important findings of these 20 interventions. The list of the 20 most promising interventions comprised 7 scientific publications and 13 internet findings of which 6 were clustered selections. Clustered interventions refer to more than one intervention/application grouped under a general heading. All the interventions are listed and described as per their classification according to the taxonomy.

1. Automated Telephone Self-Management Support System. The importance of a tailored and personalised approach and an intervention easy accessible through low social and physical barriers was emphasized by Schillinger (2008) in an article referring to the IDEALL project (Improving Diabetes Efforts across Language and Literacy (USA). This tele-health, online self-management diabetes intervention had multiple complimentary components, which facilitated provider patient communication with patient self-management and health education in a supportive format (e.g. regular calls with tele-health nurse). Another strong point of this intervention is the fact that it is quite easily transferable to other contexts.
2. DeWalt (2012) showed in ‘Multisite Randomized Trial of a single-session versus multi-session literacy-sensitive self-care intervention for patients with heart failure (USA) that follow-up calls are very important especially in case of vulnerable groups with low health literacy skills. Other strong point for this intervention is the cultural adapted information.
3. In Laforest (2007) ‘I am taking Charge of my arthritis’ the most prominent aspects for the health literacy research are the components of patient-centeredness and shared- decision making of the intervention. It shows that a tailor-made approach together with home visits by a diversity of professionals is very important aspects with respect to the reach of frail seniors with arthritis.
4. Lorig et al. (2013) and Griffiths et al. (2005) showed with ‘the Chronic Disease Self-management Program (CDSMP)’ an intervention with its focus on the community (and not on the health system). It shows that an additional course on self-management (here coping skills) next to the regular treatment is very useful. The transferability of this intervention is high and the fact that they already worked with many different target groups shows that it is culturally adapted. The focus is at people with severe mental illness and the programme shows that there is a range of items (medicines, physical activity etc.) that are discussed during the course.
5. Robare (2011) highlights in ‘10 keys to healthy ageing’ the importance with respect to self- management of one to one counselling at home (personalised approach, coping skills) and trained health counsellors in the community. Also in this intervention we see efforts of changing the paradigm of medical care to more emphasis on prevention and an active lifestyle, which is a very important aspect in self-care and self-management.
6. In Pruthi et al. (2010) ‘Promoting a Breast Cancer Screening Clinic for underserved women; a community collaboration’ a plan of action for a screening program (breast cancer) is highlighted. Here taking away all kinds of barriers (social/physical/cultural) is the most important driving force together with building trust. Examples are: provision of transportation vouchers, language interpreters and use of pictorial charts. Furthermore an important aspect is that the activities are embedded in existing structures and that educational materials are adapted to the various target groups.
7. In ‘EXSOTE: Remote monitoring and Health Coaching in South Karelia (Fi)’ (Renewing Health project, June 2014) , the most important success factor in the reach of vulnerable older adults with a chronic disease is the combination of e-health and mobile techniques together with the attention of personal health coaches. This Finnish example is part of the European project ‘Renewing health’ and shows that remote monitoring of diseases should be accompanied by human support in order to be effective.
8. The ‘Talking touch screen’ , developed in the Netherlands, is an interesting example because of its user’s centred design approach: patients with low health literacy skills in physiotherapy practices were actively involved in the development of the talking touch screen; which resulted in the use of simple language, visual elements, pictures and an easy to use touch screen. With the Talking Touch Screen patients (Dutch and Turkish) with low health literacy skills are supported to fill in a patient’s specific list of complaints (PSK)) on their own. See also other example of Talking Touch Screen: Hahn et all. (2004) . In this example from the USA a multimedia program was developed to provide a quality of life assessment platform that would be acceptable to patients with varying literacy skills and computer experience. One item at a time is presented on the computer touchscreen, accompanied by a recorded reading of the question. Various colours, fonts and graphic images are used to enhance visibility, and a small picture icon appears near each text element allowing patients to replay the sound as many times as they wish. Evaluation questions are presented to assess patient burden and preferences.
9. The ‘Age action alliance’ in the UK is a supporting network/platform where many activities in the field of health ageing are being carried out. The Age Action Alliance is a network, which brings together organizations and older people, in partnership. Drawn from civil society and the public and private sectors, it takes a positive approach to ageing and seeks practical ways to improve services and support to older people. The network is aiming at a comprehensive approach regarding ageing population, where social inclusion (including digital inclusion) and participation of older adults are an important factor.
10. The voice of older people is the most prominent success factor in the ‘KOVE: Kilburn Older Voices Exchange’ project in UK. The intervention is focusing on giving older people an important voice in the community and in this way empowering them to influence the existing system. The intervention offers them the opportunity to be engaged in problem solving in the community and adjusting the environment (context).
11. The intervention ‘Erlebnis Internet’ is an interesting German example of how barriers to the access of the use of internet (digital literacy) by older adults with low SES or from other cultural backgrounds are been tackled by means of social support of peers, volunteers, tailor-made education materials etc.
In the IROHLA research more interventions on the subject of digital literacy have been found. We can mention here also the intervention ‘Electronic health information for life long learning via collaborative learning (EHILLL-CL) , which is an interesting example of collaborative learning in an informal setting. This theory-driven intervention, developed and tested in public libraries, aims to improve older adults' e-health literacy. Participants were highly positive about the intervention and reported positive changes in health-related behaviour and decision-making.
12. In the case of the online platform ‘50plusnet’ in the Netherlands strong points are the low threshold character of the intervention, the positive attitude and personal approach of peers and volunteers. 50plusnet is a social network focusing on social inclusion and the prevention of mental health. Participants of the social network can get in contact with fellows quite easily and it is possible to start up all kinds of activities together like physical activities, cooking classes, cultural activities etc.
13. Cooper et all. (2011) stress in ‘Physician communication skills training and patient coaching by community health workers’ (USA), how important it is to focus in an intervention on the communication skills of both professionals as patients. Here physician and patient interventions were designed in tandem to support the therapeutic partnership from both perspectives. Furthermore the intervention shows us the importance of the adaptation of the traditional role of Community Health Workers (CHW’s) to the role of coach—an approach that the investigators used because of the evidence for cultural relevance and effectiveness of CHWs in health education and promotion among patients from minority and underserved groups.
14. The intervention ‘Activ ins Alter; investition in die Zukunft alterer Menschen’ (AU) is focusing really at the vulnerable group of older people. It is a good example of a personalised approach and activation of elderly. The strong element is that it starts with a needs assessment; people are asked what their needs are with respect to a broad range of things (social & health infrastructures) and about the possible ways to the needed support. The intervention is promising in its outreaching character, through the use of home visits by health counsellors. Networks with stakeholders are built and awareness of the problem is raised. It is a very well described and analysed intervention. Also the evaluation on implementation, transferability, and sustainability is reported. A thorough analysis is available in the report Health pro Elderly .
15. The intervention ‘Lebenswerte Lebenswelten’ focuses specifically at older women with low SES. This community-based intervention uses existing structures and networks in the community in order to empower older people with respect to caring for their own health. Home visits by volunteers, intergenerational learning, health café’s and communication and networking amongst different organization are key elements in this intervention.
16. ‘Emerging technology for at risk chronically ill veterans’ (USA) creates a ‘Health buddy’ for veterans (mobile device). This e-health intervention is aimed at supporting patients in health problems in the home setting. The ‘buddy’ stands for having support (a friend) nearby as well as raising the feeling of security. Results are positive both at the level of users (decrease in hospital/nursing home/emergency room admissions) as for professionals. The intervention belongs to the cluster ‘E-health interventions’ with emphasis on ‘tailor-made’ and ‘interactive’.
17. The ‘Pairs program’ (USA) is an educational programme where students are getting experience with the ‘Daily living problems’ of Alzheimer patients. The project is a replica of the Buddy program, which has positive evaluations on the level of attitude (empathy) of professionals. The intervention fits to two clusters of interventions: ‘educational interventions’ and ‘Peer/social support-interventions’.
18. ‘Filmauve’ (FR) aims at informing and empowering caregivers. Relatives/care givers learn strategies how to cope with the problems of patients with dementia. It explores and educates the social context (spouse/children) of patients. Insight, knowledge and strategies are implemented in all day life of relatives of Alzheimer patients in four sessions (‘Lab’s’). The intervention is targeted at the empowerment of caregivers.
19. ‘Ask me 3’ (USA). This intervention – an educational website - is based on the principle that every patient should ask their health care providers three questions: (1) What is my main problem? (2) What do I need to do?; (3) Why is it important for me to do this?. The ‘communication between patient/client and professional’ is a relevant theme for IROHLA as it increases the Health Literacy by increasing awareness/sensitivity in both patients and professionals. This intervention has low cost for implementation, but the website should be culturally adapted to the European context. This intervention has a match with the ‘Teach-back method’ (also focussing on 2-way-communication).
20. I want to learn (I will lernen) (G). This intervention is an e-learning module, specific for people with low literacy and numeracy skills. It shows that the use of interactive websites with games/exercises (blended learning) is very relevant. I want to learn is comparable with the Dutch intervention Oefenen.nl.

3.2.4 Components of Interventions

Researchers “deconstructed” identified interventions (in the health, social and commercial sectors into components, following the taxonomy. They found in total 338 components (following the health literacy taxonomy) in 56 interventions (or clusters of interventions). Of those components, 242 aimed at individuals or their social context (family, care givers, networks, neighbourhoods, etc.) and 96 at health professionals and the health system. This was in line with the general impression that health literacy (and related) interventions principally target individuals and their social context.

Findings of the analysis were related to the previous taxonomy:
• Information and education was part of nearly every intervention. Combinations of knowledge transfer methods increased effectiveness, and repeating messages and information contributed to improvement of understanding.
• Skills’ training was an important component of health literacy interventions, for individuals and their families often concentrating on self-management skills and for professionals mostly concentrating on communication skills. A variety of methods for skills training existed, sometimes combining ICT based elements with face-to-face elements. Often effects were measured through patients’ data. Interventions focused mainly on practice, not on theoretical models.
• Behavioural change was the objective of most health literacy interventions, even if only information or skills training was involved. Behavioural change interventions in general took longer and applied multiple methods. Behavioural change theories were mentioned in effective interventions, more often for individuals than for professionals. Measuring effects ranged from self-reported efficacy to biometrical proof of adherence.
• Contextual (family, caregivers, communities) support created continuity of support in order to implement more individual health literacy interventions, and built on theories of networking and social capital. There was a clear role for ICT-based interventions in this type of activities.
• Involvement of clients in the health system was not often mentioned as component of intervention, but in several cases implicitly implemented to enhance commitment of chronic patients or health staff to interventions. Co-creation had a positive effect on acceptability of interventions.
• Removing barriers to access was rarely mentioned as explicit activity. Interventions in the social domain were often considered as helpful for strengthening health literacy interventions.
• Adaptation of existing interventions was often not explicitly identified, but probably implemented in quite a number of interventions. Electronic means of communication instead of traditional paper communication could be considered as a type of adaptation of existing interventions. Adaptation of general programmes might have played a role in individualising healthcare.

The analysis resulted in the following recommendations for health literacy programmes:
• Use a patient empowering approach and pay attention to contextual factors that could be social, physical or cultural barriers. It is clear that emphasis on self-management is contributing to better outcomes of interventions. Improving communication skills of health professionals also plays an important role in achieving patient centred health care.
• A very important aspect is the repetition of the health-related messages and use of a variety of methods of follow-up after providing health information: telephone calls, technological devices, home visits during a certain period of time.
• It is effective to offer to individuals health literacy programmes with a mix of group work and individual follow-up for at least half a year. This will result in sustainable behavioural change.
• Peer groups, buddies, volunteers, home visitors are instrumental in communication and skills (social network). Buddy-programmes and interventions for informal caregivers can help to create more knowledge and empathy on low health literacy.
• E-health interventions have to be interactive (and take a step-by-step approach) for their effects on increasing health literacy.
• Lowering or removing barriers (social, physically, culturally) is an important contribution from outside the health sector
• Community based interventions on the basis of co-creation are effective for sustainability.
Update of the IROHLA intervention model
The findings of the inventories and analysis were integrated in the IROHLA intervention model that was developed in the first phase of the project. The interventions were clustered in major groups that together result in a comprehensive approach.

Figure 2 Updated IROHLA intervention model with interventions

3.3 Validation of health literacy interventions

In the presentation of the results of the validation we follow the four target groups as identified in the IROHLA model: communities, individuals with low health literacy, professionals and health organisations.

3.3.1 Community support

From literature reviews we knew that strengthening the social support in the community, i.e. friends, neighbours, family members and other stakeholders, has the potential to support older adults with low health literacy in accessing health information and health care, and empower them in doing so. However, research on how to best promote and reinforce this process was limited. As a starting point for this, we needed to improve our understanding of the role of social factors, such as loneliness, social support, and number of social contact. Next, we needed to increase our insight in the strategies to improve support and empowerment. We conducted a quantitative study on health behaviours and social factors, a study on the perspectives of stakeholders on Community-Based Participatory Action (CBPA) initiatives, and a qualitative systematic review.

Health literacy, health behaviours and social factors; a quantitative study
Health behaviours combined with social factors may play a major role in the pathway from low health literacy to poor health, higher rates of hospitalisation, and higher rates of mortality. In the LifeLines cohort study (a large long-term cohort study into health and wellness in the Northern Netherlands) we assessed the associations between health literacy and various health behaviours and social factors among older adults. We investigated whether social factors moderate the associations of health literacy with health behaviours.
Key messages of this study are:
• Health literacy is associated with a wide range of health behaviours among older adults. This could partially explain the association between health literacy and health status.
• Low health literacy is related with loneliness, engaging in few social activities, and having a low number of social contacts.
• The negative impacts of low health literacy on health behaviours are, however, not limited to older adults who are lonely, have low social support, have few social contacts, engage in few social activities, or are living alone, but regard all older adults with low health literacy.

Application of Community-Based Participatory Action (CBPA) initiatives: perspectives of stakeholders in semi-structured interviews
Community-Based Participatory Action (CBPA) may support older adults in accessing health information and health care, but evidence lacks on the best way to achieve this. CBPA initiatives offer older adults an opportunity to reflect on their health knowledge, decisions and behaviour from their community context, through offering a space for dialogue with various community actors, such as other community members, professionals and policy makers. Guidelines on how to use the CBPA approach to strengthen critical health literacy could be useful for health intervention developers and policy makers. Semi-structured interviews were conducted with stakeholders working in the field of participatory initiatives in Italy and the Netherlands. We explored how the community was involved in various CBPA projects, and took stock of stakeholders’ needs and wishes regarding potential guidelines. Participants find it useful to have CBPA guidelines and recommended different steps in a CBPA project.
A CBPA guide for supporting critical health literacy could consist of seven steps:
1. Assess the current situation and stakeholders
2. Identify shared problems and engage project partners
3. Make a plan together with all project partners
4. Identify and design project activities
5. Carry out the project activities
6. Analyse the findings and formulate best practices
7. Evaluate and monitor throughout the project.

Improvement of Critical Health Literacy of communities: a systematic qualitative review
In earlier research we concluded that improvement of the Critical Health Literacy (CHL) capacity of communities might provide major contextual support for older adults with low health literacy. It might help to support older adults in making informed health decisions and taking actions, in their own cultural context, for the health and wellbeing of themselves and their community. Research on strengthening CHL at a community level was scarce. Therefore, a comprehensive qualitative systematic review and data synthesis was conducted in IROHLA to explore how CBPA initiatives address the CHL of older adults in their community context. We identified a set of CBPA initiatives that strengthen CHL of older adults within their community improving our understanding of how CHL can be approached at a community level.

Key messages of the systematic qualitative review were:
• A Community-Based Participatory Action (CBPA) approach in CHL initiatives has potential to strengthen Critical Health Literacy in the community.
• CHL entails:
o The knowledge, perceived understandings and related needs about health and social issues that has been developed throughout older adults’ lives;
o Collaborative learning: a formal way of health knowledge exchange amongst peers and other community members;
o Communal health actions related to wellbeing aiming at individuals and groups.

Loneliness and social isolation are important factors in low health literacy. Collaborative learning and social activation contribute to improving critical health literacy and increase the capacity of communities to handle health issues. It is important to work in a systematic and participatory manner to reach long-lasting results.

3.3.2 Empowerment of older persons with low health literacy

Interventions to enhance the capacities of older persons with low health literacy are effective, especially when these people are suffering from (multiple) chronic diseases. Below we summarise the results of intervention aiming to empower older adults with limited health literacy, regarding e-health interventions, and regarding photo stories.

Suitable e-Health interventions for older adults with low health literacy: five pilot studies
E-health tools offer major opportunities to empower older adults in gaining access to health information and health care. These tools are of increasing importance in the healthcare system: they can contribute to improvement of health literacy, self-management and healthy ageing of older adults. However, many older adults in Europe have low health literacy, which may limit their ability to work with e-health tools. A better insight into the needs and experiences of these older adults regarding e-health is required to develop criteria for e-health interventions that suit their needs. We assessed the suitability of different e-health tools for older adults with low health literacy In multiple pilot studies across Europe. We focused on different stages, from the development to the sustained use of e-health interventions.
In five pilot studies we identified the major criteria for successful e-health interventions:
1. Involvement of the target group during the development. Our results show that developers of e-health interventions for older adults with low health literacy should involve the target group in the development process to create e-health tools that are accessible (easy to use, simple), acceptable (focussed on the needs of older people) and suitable for long-term use.
2. Inclusion in existing services or familiar settings. Older adults with low health literacy are hard to reach with e-health interventions. The best way to introduce older adults with low health literacy to an e-health tool is to incorporate it in existing services or introduce it in settings in which this target group is familiar. It is also recommended to involve their closest social network in using the tool.
3. Adequate training. Older adults with low health literacy are reluctant to use e-health tool, because they often have limited experience. Therefore, it is recommended to offer training (e.g. individual support or workshops) to this group showing how to use of an e-health tool.
4. Tailoring. E-health tools for older adults should be tailored to this group. Older persons prefer simple and goal-directed tools, preferably in combination with sufficient personalisation, so that users can work on their own personal goals.
5. Sustainability. Finally, specific action should be taken in order to motivate older adults with low health literacy to maintain using an e-health tool for an extended period of time, for example by continued feedback, counselling or support.

Photo stories to empower and support older adults during their primary care consultations: development and testing
The quality of doctor-patient communication has been shown to be one of the essential factors in the process of health care and is related to health outcomes, adherence and patient satisfaction. However, older adults with low communicative health literacy perceive barriers in the communication. This negatively affects outcomes such as health knowledge, understanding, decision-making and self-management. In the literature review in WP2 we concluded that the use photo stories might be an important strategy to empower and support older adults in the communication with their health care provider. Photo stories may advance access to health care of older adults with low health literacy. Successful doctor-patient communication relies on appropriate levels of communicative health literacy of older adults, the ability to deal with information and communicate about health.
We first developed a narrative- and picture-based health literacy intervention intended to support older patients with limited health literacy during their primary care consultations. This yielded seven photo stories incorporating principles from narrative and social learning theory and covering communication themes and strategies identified during focus group discussions and role play exercises. The intervention was developed into three different formats: one-page photo stories, narrated video-clips using the original photo story pictures; and interactive video-clips covering participation and communication during primary care consultations. Older adults in the pilot evaluated the intervention positively.
Second, we performed several studies evaluating the effectiveness of the different formats of the photo stories. Video clip photo stories on doctor-patient communication make this intervention accessible for people with very low reading skills. The first study concerned a two-armed randomised controlled trial (RCT). We assessed whether the addition of interactivity increases the effectiveness and/or appreciation of a set of online photo stories targeted to people with lower levels of health literacy among 197 users of the Dutch adult learning website “oefenen.nl”. This RCT showed no significant differences for both interventions with respect to self-efficacy, behavioural intentions and satisfaction of participants. However, in accompanying focus group discussions participants with low health literacy did express a strong preference for the interactive intervention. The participants found interactive photo stories more effective and appealing than non-interactive photo because interactive stories increase self-referencing and active processing.
The second study to evaluate the photo stories consisted of three parts, regarding effectiveness, appreciation and cultural appropriateness of the photo story intervention. These parts were (1) an RCT in Germany (N=126) and (2) a comparison of a photo story booklet with a traditional brochure in the Netherlands during interviews (N=13) and (3) a focus group discussion exploring Italian older adults’ opinions on the intervention in Italy (N=7). In the German RCT, no significant differences were found in communication self-efficacy or appreciation when the photo story booklet was compared with a traditional brochure in Germany. Unfortunately, the German sample group had on average relatively high levels of health literacy and communication self-efficacy, making it harder to detect any changes or differences in self-efficacy.
The comparison of two formats presented to older adults in the Netherlands showed that photo stories on paper were evaluated more positively than interventions on a tablet. Participants in the interview study in the Netherlands preferred the photo story intervention to the traditional brochure.
The cultural appropriateness of the photo story intervention was assessed in a focus group study in Italy. The older adults considered the photo story booklet as a valid and helpful intervention. In most aspects the intervention met the needs of the Italian participants. The main messages and scope were evaluated positively but a number of important culture-based issues should be addressed.
Overall, photo stories about doctor-patient communication that are co-created with the target group are a promising intervention, evaluated positively by the target group. They offer real-life step-by-step scenarios that help form behavioural intentions and communication strategies. Cultural adaptation of the health literacy intervention to other European countries may be necessary to maximize relevance and impact.

Innovative approaches in empowering older adults and improving their health literacy are promising. Interactive and tailor-made, culturally sensitive tools can reach older adults, who may not be easily reached by tools aiming at younger people.

3.3.3 Enhancing the capacities of health professionals in communication

Training health workers in health literacy focused approach: a two-staged approach
Enhancing the capacities of health professionals in communication with older adults with low literacy may mitigate the adverse effects of low health literacy, as shown in reviews in WP2. Health professionals are in a unique position to mitigate the effects of low health literacy. Effective communication between health service users and healthcare providers is associated with improved patient health outcomes, and with patient safety. It is one of the most important contributors to patient engagement, participation and adherence to healthcare plans. Qualified health professionals can lack awareness and understanding of health literacy issues, as well as skills and abilities to address low health literacy effectively. A need for sustained training for health professionals in health literacy principles has been recognised with the identification of educational techniques and tools.
A stepwise approach was used to inform the development of an evidence based training programme in health literacy focused communication for health care professionals in Europe. First, we performed a study identifying the experiences of health professionals in Ireland, the Netherlands and Italy in communication with patients with low health literacy. This was in preparation of a format and content of an evidence based pan-European training in health literacy focused communication skills. Second, we developed and assessed the feasibility and acceptability of this training.
Key messages of these studies are:
1. Health professionals working in three EU Member States experience health literacy as a problem in daily practice.
2. A need exists for multi-disciplinary health literacy communication training for qualified health professionals working in practice in Europe.
3. Health professionals prefer a blended learning approach to continuing professional development with a combination of didactic and experiential teaching strategies with explicit links made to practice.
4. Core components of the modular training programme in health literacy focused communication are knowledge of health literacy, identification of health literacy problems and interpersonal communication strategies to facilitate the development of functional, interactive and critical health literacy skills of patients.
5. Health professionals prefer experiential learning in a safe and supported environment, using role-plays and recorded interactions and perceived that this enhanced their skills in health literacy focused communication.
6. Training in health literacy focused communication as developed in IROHLA in three European countries is feasible and acceptable to health professionals across health settings and in multiple disciplines. Learning outcomes are promising and the training increased awareness, knowledge, self-efficacy and skills of professionals in a simulated practice session.

Training professionals is useful for enhancing health literacy outcomes, and well-received if offered as modular blended learning. Training programmes can be similar in different countries in Europe.

3.3.4 Health Literacy Friendly Organisation: Improving the accessibility of health system for older adults with low health literacy

Environmental Health Literacy Interventions: two studies
In WP2 we concluded that making health organisations more health literacy friendly is a major support for health professionals in adequately providing care to older adults with low health literacy. Environmental barriers impact on the participation and quality of life of patients with low health literacy. Environmental Health Literacy Interventions (EHLIs) may be an important strategy to reduce potential barriers for people with low health literacy. EHLIs identify and direct action to reduce these barriers in health settings, improving access and self-management, particularly for those with low health literacy. However, evidence on their use and effectiveness is scarce in the European Union.
In two studies we identified core components of EHLIs, assessed the perspectives of international stakeholders on the content, approach and application of EHLIs and evaluated the implementation process these interventions in healthcare settings in three European countries.
Key messages and conclusions are:
• Across countries and in all healthcare settings, health literacy was recognised as a critical issue to be addressed at the organisational level.
• The majority of the thirteen identified EHLIs are designed for general healthcare settings and can be used by health professionals of various disciplines.
• Existing EHLIs do not take adequately account of preventing high risks and patient safety. More attention must be paid to recognising high-risk situations for patients such as prescribed medications, care transitions and hospital discharge.
• Involving patients with limited HL in EHLIs deserves more attention, as this was currently only the case in two EHLIs but is generally considered to be important.
• The implementation of EHLIs contributes to the identification of health literacy problems in the environments that are amenable to change with forethought and active planning.
• Active participation of service users and people with low health literacy, staff and managers is necessary for EHLIs to be successful in identifying and reducing health literacy barriers in healthcare environments.
• EHLIs are complex by nature and thus made up of multiple components. The selection and order of delivery of components of health literacy environmental interventions can differ and more research on optimal implementation is needed.

Environmental Health Literacy Interventions contribute to creating health literacy friendly organisations, where professionals can support people with low health literacy. These interventions can be effective if custom-made (also for hospital settings) and designed in a participatory way.

3.4 Producing guidelines for policy and practice

The aim of the IROHLA project from the onset was to translate research findings into implementable recommendations. Therefore part of the research work was how to present the recommendations from research in a format that would be attractive for the broad range of stakeholders.
Guidelines were formulated for three levels: the policy level for strategic choices in health literacy issues, the guidance level for designing evidence-based interventions and the practice level, showing best practices in health literacy interventions.
The basis for the guidelines was provided by the previous work packages of IROHA. All results of the previous activities of formulating a framework, making an inventory and validation activities contributed to the development of the guidelines. We translated the available evidence into concrete advice for policy and practice. We used the IROHLA model and the taxonomy, as well as key elements of the 20 most promising interventions identified as the basis for the guidelines. We added elements and suggestions based on review of the existing policies and strategies, the suggestions of panellists, and reviews of recent publications.

Figure 2 Process of formulation of recommendations IROHLA

3.4.1 Policy briefs for EU member states (policy level)

Multiple methods were used for the systematic development of the guidelines for policy, such as reviews of existing policy documents, consultations and discussion meetings. An online survey was conducted about relevant questions to be addressed in the guidelines, and contributing factors to successful dissemination and adoption of guidelines. Key messages for the guideline for policy were pilot tested during face-to-face interviews, online polls and conferences. Draft policy briefs were presented to stakeholders and tested for comprehension and acceptability.

Key findings of the activities:
• Relevance of the policy briefs: briefs should respond to a felt need of the target audience, or in other words address actual problems encountered. In framing the issues the daily concerns of the audience should be addressed, e.g. accessibility of health care, quality of service delivery.
• Adequate solutions: briefs should be offering solutions that are within reach of the target audience, and should not aim for the highest (not-attainable) goals. Offer practical solutions, e.g. link health literacy to other equity-oriented policies or to informed consent regulations, rather than producing something new from scratch.
• Feasible actions: briefs should show evidence-based feasible actions, which produce quick wins and lead to further action rather than proposing long-lasting investments.
• Local adaptation: briefs should be offered a way that enables local changes as required by the context. General principles should be clear and opportunities for adjustments should be pointed out.
• Follow up: presentation in the right forums, and personal follow-up with decision-makers is necessary
Based on the recommendations and research the final policy brief for EU member states was produced and disseminated.

3.4.2 Policy briefs for health organisations (guidance level)

Based on literature studies we developed a framework that organisations can use to develop interventions. This framework explains relationships between action and outcomes.
In concrete recommendations this means that developers of interventions should take into account:
• Context: interventions do not take place in isolation, but within a community, organisation or society. This context must be analysed every time again. Simply copying or duplicating is not an option. Social networks, power relations and cultural values have to be taken into account.
• Intervention mechanisms: interventions consist in general of a complex set of activities, e.g. aiming at individuals affected and their families, at health care workers and their organisations, or at information, skills building and behavioural change. The interventions must be “deconstructed” and working mechanisms of composing activities have to be identified. In scaling up or replicating interventions suitable working mechanisms have to be put together in a workable composition (this was elaborated in WP7 of IROHLA). Knowledge gained in studies into implementation fidelity is applicable here.
• Action theory: in design of interventions evidence-based theories must be applied. In public health enough evidence has been built around interventions that can be used. In WP2 an inventory of these applicable theories (constructs) was produced.
• Concept theory: concept theories look at the theories of change: how does a series of activities or interventions lead to impact. What are the pathways to change? Simple health education will not result in healthier older adults; better information for health workers will not lead to HL friendly institutions. This is very much about embedding interventions in a wider range of actions to achieve a final impact.
We also developed a stakeholder analysis of interested parties, which should be targeted for dissemination of the guidelines for policy and practice. We decided to identify as target groups, policy makers, healthcare and social welfare practitioners, researchers and educators. Patients and community organisations were also seen as beneficiaries of the research.
The policy brief for health organisations provides recommendations on basis of the findings in the research and consultations.

3.4.3 Guidelines for practice (service delivery level)

Through consultations and interactions with practitioners we formulated a plan for presentations of the guidelines at the service delivery level
• Practitioners are interested in a broad range of health literacy interventions targeting the older adults, professionals and/or their context.
• The recommendations for practitioners should suit their specific needs and circumstances.
• Recommendations for practitioners should be evidence-based, and presented in an unambiguous way (with references to background reading)
• The best way to present recommendations is in a flexible web portal, where a user-friendly search engine leads interested people to the needed information.
IROHLA developed www.healthliteracycentre.eu where interested professionals and policy makers can easily identify information they need. UMCG and partners will guarantee future updates of the portal with new information.

Table 6 Table of Content of web portal for IROHLA results

Content Sub-level content

General, A) Text and slogan landing page
General, B) Film
General C) Introduction
General D) How can we promote health literacy? (different levels, focus on the 'how' for each level)
General E) Context information or 'about' ('context' is not a good word)
General F) For who is this portal: for each target group: what is there for them?
General E) Short content for older people with low health literacy
Key messages Practice
Topic 1: Health literacy Defined and Measured
1A) Understanding health literacy
1B) Improving health through health literacy for older people
1C) Recognize low health literacy
1D) Low health literacy in Europe
1E) Measuring Health Literacy (different concepts).
Topic 2: Health Literacy: Promotion, Prevention in Healthcare and Social Services
2A) Quality improvement of healthcare and social services Integrating health literacy in quality of care approach, as part of health systems.
Patient safety is a system property and the foremost attribute of quality of care.
Health Literacy Environment assessment tools should be more comprehensive, facilitate practical application and aim at mitigating negative effects of low health literacy.
Health promotion and primary prevention in training and education for health and social staff. Make them aware of their influence.
2B) Accessible health- and social services Easing access to health and social services improves health literacy
2C) Patient centred care Care should promote self-management, not dependency (encourage asking questions, teach back, motivational interviewing, role of culture were doctor knows best; be explicit on expectations and responsibilities).
Determinants of effective shared decision making like personalized health care, building up a trustful relationship, open attitude, treating older adults with respect and dignity.
Role of social networks and social support.
2D) Self-care and e-health and M-health Make it innovative and user-friendly.
It should give users a feeling of privacy and reliability.
Use tools to measure needs and wishes and to evaluate e-health interventions for our target group (questionnaire WG2 and the eASI).
Aspects of effective E-health and M-health interventions.
2E) Health Literacy and Adherence Improvement of health literacy is a result of interaction between healthcare professionals and citizens (topic for national and local health systems).
Topic 3: Health Literacy: Communication and Empowerment
3A) Health communication Building on strengths of individuals (acknowledge expertise of people with chronic condition). To enable older people to stay independent for a longer period of time
Various routes and methods of communications (face-to face, pictures, text, graphs, drawings, film, phone, Internet).
Positive message-framing is more effective.
Motivation and realistic goal setting. Step by step approach
Communication with focus on low literacy patients
3B) Community Participation in Health Engage communities to facilitate individual’s health
Mutual support (partner up with the community, facilitating co-creation, engaging community leaders)
Addressing loneliness (important factor in poor health and low health literacy)
3C) Successful interventions in Communication and Empowerment Comprehensive health literacy interventions and -policies are more effective if the combined interventions address knowledge, skills, and attitudes of all stakeholders within their context and also reduce barriers to access of care.
Selected interventions IROHLA: description and links to more information (evidence for added value of comprehensive approach).
Tools on specific themes.
Checklist on health literacy -interventions with barriers and facilitators for implementation (contextual factors, co-creation, implementation fidelity).
Topic 4: Health Literacy Policy, Management and Development
4A) Policy on Healthy Ageing integrating health literacy
4B) Enhance development and implementation of guidelines, policies and interventions Introduction on Guidance
Health literacy policies are more effective if they give priority to interventions that are based on a proven theory of change, applying scientific approaches in achieving personal and organisational change.
Adapting to specific contexts and co-creation (context sensitivity).
Implementation fidelity.
Continuity of care: promote interventions that are fully integrated in healthcare, belonging to standard operations, and part of standard curricula for training professionals.
Interdisciplinary linking: interventions in the social sector can support health literacy policies. Use and apply the opportunities and lessons learned from other sectors.
4C) Health Literacy and Social Inclusion Interventions in the social sector can support health literacy policies by promoting non-healthcare-related interventions that facilitate access to health and social services. Make health literacy part of social inclusion.
Topic 5: Research priorities
5A) We need a long term view (vision) on research with vulnerable groups
5B) More evidence is needed about cost effectiveness
5C) Include a broader range of skills in research on Health Literacy
5D) Find effective ways of Information Dissemination (for professionals and health sector institutions)
5E) The prevalence of limited health literacy in specific populations and their needs in care and support.
5F) Themes from meeting June 10th :
1) Focus on intervention research
2) Understanding the health literacy concept and relationships with a comprehensive research approach
3) Understanding comprehensive interventions
4) E-health and M-health
5) Communication and empowerment
6) Diversity and personalised approach
7) Implementation research
8) Development of methodology: intervention research
9) Life course perspective

All scientific results of IROHLA were translated into concrete advice for policy and practice. Engaging stakeholders in the developmental process was crucial for identification of key issues to be addressed, for finding viable interventions in different cultural contexts and for presentation of the evidence in policy briefs and in a user-friendly and sustainable web portal that meets the needs of end-users. This will create awareness and support them in producing effective strategies and interventions to improve health outcomes among older adults with low health literacy in Europe.

Potential Impact:
4 Impact

4.1 Expectations in the project proposal

The IROHLA project planned to enhance healthy ageing through improvement of health literacy in Europe. The guideline for policy and practice (deliverable of this project) should provide concrete tools for national, regional and local government authorities to start action. The (maximum) 20 evidence-based interventions for empowerment and health literacy practice should be applicable in all European member states, as they will be selected on evidence and feasibility.
The participation of representatives of the ageing population in the project, local governments, business community and other stakeholders would guarantee that the feasibility of selected interventions was high. The extensive network of organisations involved in the project should guarantee that information could indeed available to all stakeholders, who bear responsibility for healthy ageing programmes.
The IROHLA consortium indeed managed to produce guidelines for policy and practice formulated in policy briefs and presented in an interactive web portal Health Literacy Centre Europe. Interventions were selected and validated in a variety of European contexts. Recommendations were formulated in consultation with stakeholders, and pre-tested in a group of policy makers.

4.2 Health literacy in the European Union

4.2.1 Key messages for the EU in IROHLA policy briefs

• The European Union aims for effective, accessible and resilient health systems in Europe. Healthy ageing is an important priority given the demographic changes in Europe. The EU foresees changes in the health systems to address issues of ageing. Health literacy for older people can be incorporated in policies and action programmes like the European Partnership for Active and Healthy Ageing (EIP-AHA) has already done.
• The European Union promotes e-health and m-health as part of the Digital Agenda. The health literacy criteria developed by IROHLA for usability, usefulness and sustainability provide critical advice for all IT applications, as they will enhance utilisation by all groups in society. Regulations and inter-company standards will contribute to more effective use of these innovations.
• An increasing body of knowledge on health literacy in Europe asks for EU-guided actions for scaling up and for monitoring implementation of health literacy policies and action plans of member states.
• Research and development in the area of health literacy during the life course will contribute to the healthy ageing of the European population. Further research into cost-effective interventions is necessary to improve the sustainability of health systems.

4.2.2 Expected impact in the EU

Healthy ageing is more and more an EU priority and in the European Partnership for Active and Healthy Ageing IROHLA consortium partners contributed in various working groups. Health literacy is featuring prominently in action plans, for example in the area of prescription and adherence to medical plans. Validated tools and instruments identified in the IROHLA project have been shared with EIP-AHA. More tools are available for sharing with stakeholders via de web portal.
By continuing sharing knowledge and tools with other organisations at European level, former IROHLA consortium partners will continue to give direction to the development of health literacy interventions. The pathway from health literacy interventions to health ageing is becoming more and more clear in international publications, also in the EIP-AHA. Improved self-management and improved control of chronic diseases, better adherence to medical plans, these are all proven effects of health literacy interventions. The process to healthy ageing and increase of life expectancy as advocated by the EU is clearly laid out in the EIP-AHA strategy. It is very encouraging that this powerful network is taking the lead in policy development for healthy ageing in the EU and incorporating health literacy.
In June 2015 the IROHLA project was awarded the Grundtvig Award by the European Alliance of Adult Education (EAEA) because of its efforts to improve literacy-related capacities of people. EAEA had put health central in its 2015 programme and had commissioned research into the relation between adult education and health. In this research it showed that improved literacy had a direct effect on health. EAEA will continue to advocate for health literacy in its future activities.
The digital agenda with e-health and m-health features prominently on the EU priorities. E-Health and Ageing is a special area in that agenda. Lessons learned from various validation activities in the IROHLA project have been shared in various forums and are prominent on the Health Literacy Centre Europe web portal. (See annex for infographic on design of e-health interventions.) E-health for the ageing population will continue to be a priority area. In DG Connect there is a strong basis to continue advocating for digital health literacy.
In the IROHLA consortium there were four small businesses involved in the development and validation of e-health tools. Those have now improved products that they will continue to use in practice. These are in the area of health information for ageing people, quantified self and self-management, interactive consultation tools and self-directed e-learning.
In the wide area of health literacy strategies and action plans the EU needs to take its place in monitoring and evaluation of activities. The IROHLA project has made an inventory of interventions and of policies in a wide area of health literacy, empowerment, and equity. The short-term HEALIT4EU project launched by DG Sante in 2014 made an inventory of health literacy interventions and policies in a much more restricted area . The IROHLA project provided an input to this report. Hopefully, there will be a follow up after this inventory, and will involved DGs continue to take their responsibility in monitoring. The Health Literacy Centre Europe will continue to provide updates in its portal, accessible for all interested parties.
Research and sharing knowledge remains important in the young research area oh health literacy. The IROHLA project helped the health department in DG Research in organising an exchange among research project staff involved in patient empowerment in January 2015. This exchange resulted in continued collaboration with other projects and joint organisation of further activities, e.g. the third European Health Literacy Conference. Project coordinators from IROHLA and Diabetes Literacy were leading that conference and will continue to work closely with the research community in health literacy to follow up on results achieved in the projects.
In the mean time, the research agenda in Europe is developing in Horizon 2020. Health literacy is a small but relevant part of the agenda.
Unfortunately, the IROHLA project did not find enough publications showing the cost-benefit analysis of health literacy interventions in Europe. We still depend on such analysis done in the United States and Canada. There is a firm belief that improved health literacy in the ageing population will reduce costs of healthcare and will contribute to healthy ageing. Further attention for cost-effectiveness analysis of health literacy interventions should be put high on the research agenda. That will complete the picture of evidence-based interventions that are valuable in the EU.

4.3 Health literacy in national policies

4.3.1 Key messages for member states in policy briefs

• Many EU member states have developed specific policies and action plans for health literacy. Other countries could follow their example and take lessons learned from those policies as described by IROHLA. The EU could support this by systematically monitoring the implementation of these strategies. Measuring the effects of interventions within a framework will contribute to better understanding and further development of the comprehensive approach and will allow for creation of the synergies between activities.
• Health literacy can be integrated in all national and regional policies and strategies that focus on quality and equity in health or on health promotion. Improving health literacy will not lead to further increase in costs related to the health of the ageing population.
• Integrating health literacy interventions in programmes for healthy ageing will empower older people to manage their own health. There is room to improve healthy ageing strategies in many member states. Health ageing will enable older people to participate in society for a longer period of time and contribute to the economy and wellbeing.

4.3.2 Expected impact in member states

The number of EU member states that are producing health literacy policy documents, strategies and action plans is increasing rapidly. Already nine countries have documents, which include attention for health literacy interventions (not necessarily only focusing on health literacy). That is a very positive development. IROHLA consortium partners manage to reach at least 12 EU member states and will continue to lobby for further development of policies and action plans. The policy briefs for EU member states have been disseminated widely and can reach many government officials and policy makers through the networks of IROHLA partners. More than 1,000 copies of the policy briefs have found their way to government officials, as well as thousands of persons who received the policy briefs by mail.
One important conclusion of the IROHLA project was that health literacy could be incorporated in so many other policies, e.g. with regard to access to care, patient safety or quality of care. Member states interested to promote health literacy for healthy ageing have a wealth of possibilities to incorporate it in standing legislations and regulations. On the Health Literacy Centre Europe good examples are shown.
It is important to mention here that promotion of health literacy is not restricted to the health sector. From the onset the IROHLA consortium has opted for an inter-disciplinary approach. Not only health organisations, but also communication experts, social services experts, health insurers, businesses participated in IROHLA. This enabled that we put health literacy in a broader context. It has yielded results. There are in the social sector many occasions to support health literacy strengthening. In the list of 20 examples activities in the social sector are listed. Social cohesion, community participation, reducing loneliness, are all examples of areas that have a direct impact on health literacy and healthy ageing. Computer literacy, language courses, literacy courses have direct effects on understanding and on abilities to manage health. Reducing barriers in access to facilities and institutions, mobility and communication enable participation and possibilities to improve health literacy.
The main message from IROHLA to the EU member states is that health literacy is not rocket science and integration of effective elements in existing programmes and systems can go a long way in achieving better health for the ageing population. It is important to draw lessons from other sectors and to work as interdisciplinary teams on health literacy and related topics. The examples that IROHLA brought together offer abundant illustrations to this effect.

4.4 Health literacy in health organisations

4.4.1 Key messages for healthcare organisations in policy briefs

• Integrating health literacy in quality of care programmes in health institutions will contribute to their success. A personalised patient approach will lead to better adherence to medical plans and improved self-management. Health literacy will make informed consent more meaningful. It will increase the overall quality of health services.
• There is a clear business case for health literacy: more efficient use of health services and better adherence to prevention and treatment will reduce the overall costs of health care.
• Improved knowledge of health literacy in health organisations and enhanced capacities of health workers will lead to better work satisfaction and efficiency of the workers. Therefore integrating health literacy in medical and paramedical training programmes will be effective.
• Empowering older people and their social environment is the mandate of community organisations. Capacity building for health literacy as part of social welfare programmes is an example of introducing health in all policies.

4.4.2 Expected impact in healthcare organisations

In organisations in many EU member states there is increased attention for health literacy of the older population, even if it is not always labelled like this. Healthcare providers, health insurers, patient organisations are all involved in improvement of health literacy and self-management. One of the examples is the European Patients’ Forum that has launched the campaign in 2015: patients prescribe 5e (self-efficacy, self-awareness, confidence, coping skills, health literacy). This European platform of patients’ and consumers’ organisations can create leverage in raising awareness by integrating health literacy in a set of synergetic measures.

Presentation of evidence-based tools
The IROHLA project met enthusiastic responses from patients, healthcare workers and healthcare organisations in many of the validation activities of evidence-based interventions. For example the photo novelas drew wide publicity, whereby patient information was presented in comic strips. Various organisations have already indicated they will introduce these in daily practice, and even nurse training schools have indicated that they want to use it in pre-service training to instruct nurses in communication.
The initial idea of IROHLA was to produce a book with evidence, but this type of presentation was abandoned and replaced by an interactive web portal where all tools and best practices are presented. Over 200 web pages have already been created and more will follow. The 20 most promising interventions have been described in detail, including contact addresses of developers and further developments after publication of those interventions. The Wikipedia structure allows for updating the website, with inputs from collaborators worldwide. The impact of the website cannot be measured immediately after closing the project, but will be long lasting as the UMCG and RUG have made financial commitment to maintenance of the web portal.
Areas where impact can be expected:
• Development of e-health and m-health: the project has developed a clear set of guidelines for the development of internet-based tools for communication and for activation of older patients and citizens in general. With these guidelines developers of apps will be able to make better, user-friendlier products, that will be used for a longer time than apps developed in the past.
• Photo novelas: the comic strip and its interactive web-based variation was a publicity-generating hit in the IROHLA project, as mentioned above. It will surely be used in many health organisations. It has been tested and it is usable in many cultural contexts. IROHLA consortium partners have already made plans for further dissemination of the materials in other countries.
• Community based interventions: The IROHLA project developed guidelines for community-based action to strengthen critical health literacy. These have been formulated is practical instructions that health and welfare organisations can use to implement their work more systematically and more effectively. The organisations in the consortium with community networks (e.g. BRHCA and AGE) will disseminate the guidelines in their networks and reach hundreds of organisations.
• Training health workers in communication: The training programme in health literacy focused communication was well received in the validation period. The programme can be used in many European countries with little modification. Presentations during conferences and publications in newsletters and newspapers resulted in immediate responses and calls for information. The training modules are now further developed and refined and will be made available through the web-portal. Several institutions in the consortium will integrate these modules in their training programmes. It is expected that these training activities will reach thousand of practitioners in many countries in Europe.
• Tools for testing health facilities: The IROHLA project validated tools for assessing health literacy friendly environment for health organisations. Those tools were also shared with the EIP-AHA and were published on the website. The tools allow organisations to test how accessible they are for people with low literacy skills. Disseminating the tools will lead to more and more health literacy friendly health organisations and therefore reduction in barriers to those organisations.

4.5 Expected impact on business opportunities

The project has taken stock of ICT solutions that could help improving health literacy. The inventory in the IROHLA project showed a wide variety of e-health applications for enhancing health literacy. The project validated various ICT tools and produced a guideline for developing apps targeting older adults with low health literacy. This guideline was made available through the health literacy centre portal and will enable businesses to be more successful in reaching older adults with their products.
In the IROHLA project a variety of ICT tools was developed and validated:
o Tool for coaching healthy lifestyles, through monitoring physical activities, diet, providing feedback and advice.
o Tool for interactive group coaching, helping individuals to improve their lifestyles.
o Tool for monitoring physical activities and fitness of admitted patients
o Tool for home-based communication, ordering of home-based medicines, and monitoring fitness
o Website with information for older adults on nutrition.
See for details of products exploitable foreground.

List of Websites:
www.irohla.eu
www.healthliteracycentre.eu

Principal Investigator: Prof. Dr. S.A. Reijeveld, s.a.reijneveld@umcg.nl
Project coordinator: Dr. J.A.R. Koot, J.A.R.Koot@umcg.nl
final1-brochure_organisations.pdf
final1-irohla-publications-done-and-planned.pdf
final1-brochure_making-of---.pdf
final1-brochure_policy.pdf