PDF to HTML - Convert PDF files to HTML files

Europaudvalget 2011-12
KOM (2011) 0788
Offentligt

National report on the

Lifelong Learning Programme (LLP)

implementation 2007-2009

in Denmark

INTERIM EVALUATION

May 2010

PDF to HTML - Convert PDF files to HTML files

Index

Executive summary..................................................................................................................................3

Context and methodology..............................................................................................................5

1.1 Introduction ..........................................................................................................................5

1.2 Context and objectives of the report ......................................................................................5

1.3 Methodology.........................................................................................................................6

Relevance, effectiveness and efficiency .........................................................................................9

2.1 Relevance .............................................................................................................................9

2.1.1 Contribution to national policy priorities ......................................................................9

2.1.2 Added value for target groups ....................................................................................10

2.1.3 Added value for organisations....................................................................................12

2.1.4 Long-term effects ......................................................................................................13

2.2 Effectiveness.......................................................................................................................14

2.2.1 Integration of previous programmes into one LLP......................................................14

2.2.2 Information and consultation procedures....................................................................15

2.2.3 Application and reporting procedures.........................................................................15

2.2.4 Recruitment of participants ........................................................................................17

2.2.5 Participant profiles.....................................................................................................18

2.2.6 Dissemination of LLP results .....................................................................................19

2.2.7 Visibility of the LLP in the education and training community...................................20

2.3 Efficiency ...........................................................................................................................20

2.3.1 Cooperation between authorities ................................................................................20

2.3.2 Management supporting tools ....................................................................................22

2.3.3 The level of financial support.....................................................................................22

Conclusions and suggestions for future developments.................................................................24

3.1 Conclusions ........................................................................................................................24

3.2 Suggestions for LLP 2010-2013 ..........................................................................................25

3.2.1 Flexibility and synergies ............................................................................................25

3.2.2 Administrative cohesion ............................................................................................26

3.2.3 Systemic impact ........................................................................................................27

3.2.4 Information and dissemination...................................................................................27

3.2.5 Inclusion....................................................................................................................28

3.2.6 Quality and qualifications ..........................................................................................28

NIRAS

PDF to HTML - Convert PDF files to HTML files

Table 1: Survey respondents’ distribution on programme and sectoral programme. Percent of responses . 8

Table 2: To which extent do you find that the young people have improved their qualifications within the

following areas through the Programme? ...............................................................................................11

Table 3: To which extent do you believe that the participation of the institution/organisation in the project

in which you have participated … .........................................................................................................13

Table 4: How satisfied have you been with the administrative procedure in connection with the

following? .............................................................................................................................................16

Table 5: From which region do the participants primarily come?............................................................18

Table 6: Which special needs have there been? ......................................................................................19

Table 7: To which extent do you find it likely that the activities completed in the project would have been

completed, had there been no grant schemes under EU educational programmes? ..................................23

NIRAS

PDF to HTML - Convert PDF files to HTML files

Executive summary

This report sums up the conclusions of the interim evaluation of the Lifelong Learning Programme

from 2007 to 2009. The evaluation was requested by the National Authority for the Programme, the

Danish Ministry of Science, Technology and Innovation, which by the European Commission was

commissioned to complete an interim evaluation according to the guidelines offered by the Commis-

sion. The evaluation has been conducted by NIRAS A/S from February to May 2010.

The evaluation shows that a vast majority of project managers judge the activities for which support

can be applied through the LLP to

be relevant in respect of the requirements of their organisa-

tion/association.

Almost eight out of ten project managers assess the LLP Programme to be

comple-

mentary with Danish national activities and policies in the area.

Most of the target groups (70 %)

have had aspects of mobility, such as study, traineeship or exchange visit included in the activities.

Most project managers and project participants assessed these mobility activities to have the appropri-

ate length, and the vast majority of the project managers are also satisfied with the contents and out-

come of the mobility activities. The results further show that the LLP Programme creates considerable

added value – both at target group level and organisational level.

At target group level project managers as well as project participants assess that the LLP activities to

great extent improve the professional and personal qualifications of project participants.

Es-

pecially ‘intercultural competence’ is emphasised at participant level as well as personal matur-

ity/development, communication, language, cooperation and networking skills. Furthermore, the vast

majority of both project managers and project participants assess that the target group will benefit

from the experience in their future education and/or profession.

At organisational level the evaluation shows that the Programme to a high extent has

added a Euro-

pean dimension to the institution/organisation.

For instance, eight out of ten project managers

believe that the Programme has contributed to the establishment of new partnerships with other insti-

tutions/organisations. This is the general picture across sectoral programmes; however, especially

project managers within Erasmus and Leonardo have this conviction. Partnerships contribute to an

exchange of experiences with other institutions or organisations across borders, e.g. in relation to

work methods, development of education modules, courses across borders etc. What should also be

stressed is that almost nine out of ten project managers

believe their international cooperation rela-

tions to endure after project termination.

Thus, the results indicate that the Programme has a posi-

tive international outcome by increasing cooperation with other institutions and adding international

dimensions and perspectives to the organisations/institutions. Equally important, the study shows that

in general it is possible for project managers to adapt LLP project activities to other activities of the

institution/organisation.

The overall relevance of the LLP is closely connected to the long-term effects of the programme. The

survey indeed shows that project managers in general judge the LLP to provide also long-term posi-

tive effects on the institutions/organisation. For instance, most project managers believe that project

participation will encourage increased participation in other international activities/projects under the

EU educational programmes as well as increased participation in other international activities/projects

in general. Long-terms effects in terms of increased participation in national activities and projects

seem to be slightly less common. The study outlines a number of important factors in order to claim

maximum benefit from the Programme. Particularly, project managers address the need of experience,

that is, experience to fill in the forms of application as well as experience in managing international

projects and partnerships. Further, the importance of having continuance in the staff handling the

projects is underlined as well as the importance of staff driven by determination and inclination.

The survey shows that the creation of a single LLP Programme, with sectoral programmes targeting

the different sectors of the educational world has not had any noticeable impact on the target group.

Most project managers seem not to have noticed the merger and/or have not felt a difference at all.

However, the dialogue meeting and the qualitative interviews indicate that beneficiaries across the

sectoral programmes would like more interaction and more possibilities to bridge the sectoral pro-

grammes. Seen from the perspective of the NA staff the integration of sectoral programmes into one

NIRAS

PDF to HTML - Convert PDF files to HTML files

LLP has been positive. It has led to more cooperation and the opportunity to relocate funds to better

achieve the goals. Furthermore, some judge the integration to have a positive effect on the activity

level within the sectoral programmes (especially Erasmus). However, some representatives from the

Danish

Agency for International Education (IU)

underscore that adequate funds have not fol-

lowed the increased activity level (especially within Grundtvig). The evaluators point to the fact that

the integration could be taken further in the organisation of the IU and in the way the sectoral pro-

grammes are implemented.

The IU provides information about the Programme, sectoral programmes and specific actions of the

decentralised parts of the Programme, gives advice to potential applicants, administers the grants and

controls and financially manages the granted projects. The information effort in general and the sup-

port and guidance provided by the IU to project managers is highly praised. Especially the very help-

ful, flexible and solution-oriented personal approach is pinned out to be very valuable. The informa-

tion by the IU through websites and written and electronic material is also seen as helpful, in so far as

it is used.

The administration and reporting procedures are some of the issues always brought up in connection

with EU programmes, and despite quite a high level of satisfaction among project managers with the

application process as well as with administrative procedures in general, there seems to be a clear

potential for improvement in some areas. Mainly the time span between the deadline for the applica-

tion and the time of approval is stressed as problematic. This specific dissatisfaction is mostly pro-

nounced for Comenius project managers. The results also indicate that there is still a huge potential

for improvements in terms of facilitating less bureaucratic and more flexible administrative proce-

dures as far as the EU regulations permit.

Even though participants with special needs are not dominant in the projects, IU representatives and

project managers (based on the qualitative interviews) seem to be very aware of the horizontal poli-

cies, that is, the LLP’s overriding purpose to increase cultural and linguistic diversity, counteract

racism, etc. there are also evidence that the international activities are starting to become a more inte-

grated part of the institutional strategy across the educational sectors.

The strengthened focus in the LLP Programme on dissemination and exploitation seem to have had an

impact on project managers. The qualitative interviews show examples of how project results are

disseminated through evaluation forms, letters and presentations by students etc. and the IU has also

done a more focused effort for dissemination of project results, through the production of a book on

creativity and innovation together with the Youth in Action Programme. The evaluation suggests that

more is done, if the necessary resources are available, to use the IU web more in presentation of pro-

ject results and products, examples of dissemination activities and FAQs about dissemination – and

about the programme in general. Finally, the evaluation points to the fact that

dissemination of the

Programme and the results of the projects could be accentuated in more cross-sectorial

events

such as the dialogue meeting arranged in connection with the evaluation.

The recommendations and suggestions for the LLP 2010-2013 and for the future programmes point to

the following aspects: Firstly, there are potential synergies in the establishment of the LLP Pro-

gramme that have not yet been realised, at the same time as there seems to be a need or a wish for

more flexibility among the users of the programme. Secondly, the administrative burden is addressed,

although this is mainly an issue with no national solution. There should, however, be a constant

awareness of this to ensure the relevance of the administration and control to the participants. At the

same time the IU should uphold the good level of guidance and service to the applicants and projects.

Thirdly, the evaluators point to the need for a strengthened focus on special needs and inclusion. Fi-

nally, the suggestions and recommendations address teacher mobility as a key area, both in stimulat-

ing student mobility and as a means to bringing internationalisation to the non-mobile.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Context and methodology

1.1

Introduction

This report sums up the conclusions of the interim evaluation of the Lifelong Learning Pro-

gramme from 2007 to 2009. It was requested by the National Authority for the Programme,

the Danish Ministry of Science, Technology and Innovation, which by the European Com-

mission was commissioned to complete an interim evaluation according to the guidelines

offered by the Commission.

NIRAS has conducted the interim evaluation during the period February to May 2010.

The European Commission has provided a set of guidelines for the report, and in order to

make the results comparable across Europe these guidelines have been followed closely. The

mandate from the Danish Ministry of Science, Technology and Innovation is a report of no

more than 30 pages, which means that not all questions have been elaborated on to the same

extent. In the Annex Report the reader will find evidence for the many conclusions in this

report.

The Danish National Agency for LLP has been very helpful in providing information on the

programme, and the participants, and the staff working with the LLP Programme have also

participated in interviews and given their view on the implementation of the programme.

The evaluators would like to express our gratitude towards all participants who at very short

notice made themselves available for the evaluation. Without their support it would have

been impossible to solve the task.

The report is structured as follows:

Chapter 1

outlines context and methodology of the evaluation.

Chapter 2

presents the key aspects in relation to the evaluation criteria; relevance, effec-

tiveness and efficiency

Chapter 3

holds the conclusions and suggestions for future development of the LLP Pro-

gramme, and future programmes.

The

Annex Report

contains:

•

An overview of the survey data in terms of frequencies and significant cross tabula-

tions.

•

An account of the activities implemented within the LLP in terms of a) the number of

applications and approved projects, b) the granted funds, c) the nature of coordinating

organisations and project participants, and finally d) the nature of granted projects. It

should be noticed that the available data is limited for the various sectoral programmes.

1.2

Context and objectives of the report

The purpose of this interim evaluation is to see whether the objectives of the European edu-

cational Lifelong Learning Programme are on course to being achieved. The Lifelong Learn-

ing Programme seeks to give the participants the possibilities to experience Europe and learn

NIRAS

PDF to HTML - Convert PDF files to HTML files

on all levels of education regardless of age, gender, role or race. The LLP focuses on formal

(contrary to the Youth in Action Programme) as well as informal education for students as

well as teachers in several institutions as for example primary schools, secondary schools,

vocational education, universities etc.

The aim of this interim evaluation is to identify ‘best practice’ and ‘lessons learned’ in this

context. The evaluation is drawn up based on three evaluation criteria outlined by the EU

guidelines; relevance, effectiveness and efficiency.

The relevance section investigates how the effort impacts on the main target groups and

organisations and whether the LLP’s objectives are seen to be relevant for the participants in

the programme.

In the efficiency section the cooperation between authorities is discussed, as well as the

management supporting tools and the level of financial support.

The section on effectiveness explores the effect of the integration of prior programmes into

one LLP, the implementation of the programme, experiences with reaching the target

groups, measures to disseminate LLP results, and finally, LLP visibility within the educa-

tional community.

Finally, recommendations for the rest of the programme period are drawn up, as well as

ideas and suggestions for future programmes.

1.3

Methodology

The evaluation is based on several different data sources. This way it has been possible to

triangulate data so that the relevance, effectiveness and efficiency of the Programme can be

evaluated on the best possible basis.

The report is based on data from:

•

Desk research on existing material provided by the Agency for International Education

(IU), e.g. data on applications and grants, data on projects and financial data etc. These

data have been categorised, organised and divided into sectoral programmes in order to

answer the questions of the evaluation

•

A quantitative survey among LLP project managers and project participants (from the

Comenius Training, Grundtvig Training and Comenius Assistantship programmes)

•

Qualitative telephone interviews with seven LLP project managers and five managers of

organisations involved in LLP

•

Two focus group interviews with project participants, one focus group interview with

project managers, and one focus group interview with programme responsible at the IU

•

Input from the dialogue meeting where almost 100 beneficiaries from all over Denmark

met and discussed the LLP Programme in sector groups and across the programme

A principal or manager of the organisation or institution that supply the participants and/or the pro-

ject manager of the activity.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Regarding the quantitative survey, the data was collected between March 11 and April 5

2010. To avoid the same respondents being asked to answer more than one questionnaire

(e.g. if the respondent was project manager in a large institution with many different pro-

jects), one project per project manager was randomly selected before sending the question-

naire. The project managers were then asked to answer the questionnaire in the context of

the specific randomly selected project.

As a result, regarding the project managers in LLP the questionnaire was sent to 745 differ-

ent projects. Of these, 361 have filled out the questionnaire resulting in a response rate of

48.5.

With respect to the participants (Comenius Training, Grundtvig Training and Comenius

Assistantship) the questionnaire was sent to 557 and with 289 filling out the questionnaire

the response rate is 51.9.

In total, 650 respondents have filled out the questionnaire. Of these 361 are project managers

(55 %) and 289 participants (45 %) from either Comenius Training, Grundtvig Training or

Comenius Assistantship. Throughout the report a distinction will be made between project

managers and participants.

The Comenius Programme constitutes the majority in the sample with more than 60 % of the

respondents.

As illustrated below, approximately 15 % of the respondents are Leonardo project managers

and around 10 % are administrators of Erasmus activities. The project managers of the

Grundtvig Programmes constitute 9 % of the sample and the LLP Study Visits constitute

approximately 5 %.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Table 1: Survey respondents’ distribution on programme and sectoral programme. Percent of

responses

Percent

Comenius

Regio

Multilateral Partnerships

Host Schools

Training

Assistantships

Leonardo

Partnerships

Transfer of Innovation

Mobility

Erasmus

Intensive Programmes

EILC

Mobility

Grundtvig

Visits and exchanges

Learning Partnerships

Training

Workshops

LLP Study Visits Programme

Total

Note: n = 650 project managers and project participants.

61.5%

0.2%

14.8%

5.4%

38.3%

2.9%

14.8%

4.3%

1.7%

8.8%

9.8%

0.9%

0.8%

8.2%

1.5%

3.2%

0.6%

4.9%

100%

For all frequencies NIRAS have analysed whether significant differences between sectoral

programmes exist. Throughout the report the most interesting significant differences are

explained and commented. All frequencies and significant cross tabulations are found in the

Annex Report.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Relevance, effectiveness and effi-

ciency

This chapter highlights the key findings related to the three evaluation criteria; relevance,

effectiveness and efficiency.

However, prior to this a very brief outline of the activities implemented in relation to the

LLP Programme is presented. A more thorough presentation of the activities implemented is

to be found in the Annex Report, section 3.

In total, 1,440 project applications were granted during the period 2007-2009. These were

followed by financial grants of 9.5 million EURO in 2007, 10.5 million EURO in 2008 and

11.3 million EURO in 2009.

There is a considerable variation among sectoral programme and actions in terms of the per-

centage of applicants who are granted financial funding and the total amount of granted

funding for each sectoral programme. For instance, within some Comenius actions around

70 % of the applicants receive funding each year, while within Leonardo mobility projects

only almost 100 % of applications were granted in 2008. The number of grants varies over

the years, just as in some years, no projects have received funding within specific actions.

Especially for Leonardo mobility, Erasmus and Comenius it is possible to provide a further

analysis based on background data from the IU. The analysis of Leonardo shows that there is

a fair regional distribution of the participants. (Based on the survey data this is also the case

for the other sectoral programmes, cf. Table 5 in section 2.2.5 )

The majority of the participants in the programme come from vocational training schools,

centres or organisations. However, a lot of different organisations are represented, from

large enterprises to non-profit associations. For the Comenius programme it is primarily

schools that get funding from the Comenius programme, whereas no non-profit associations

or NFOs or NGOs have received funding in the three years.

Furthermore, when it comes to gender distribution, the numbers for Leonardo mobility are

the only ones available. The analysis shows there is a fairly equal distribution of men and

women in the Leonardo mobility programme with a few variations within some of the sec-

toral programmes.

2.1

Relevance

The below sections describes the relevance of the LLP, more specifically its contribution to

national policy priorities, added value for participants, added value for organisations, and,

finally, project managers’ assessment of long-term effects.

2.1.1 Contribution to national policy priorities

Almost eight out of ten project managers assess that the LLP Programme either to a great

extent (49 %) or to some extent (29 %) is complementary to Danish national activities and

policies in the area. Only 3 % believe that the Programme is only to a low extent comple-

mentary to national policies (table 16 in the Annex Report).

NIRAS

PDF to HTML - Convert PDF files to HTML files

In addition, 50 % believe that the activities to a great extent give the target group some edu-

cational and development opportunities that they did not otherwise have, and 36 % believe

this to be the case to some extent. This again indicates that the Programme is complementary

to national programmes.

In general, IU representatives believe the Commission’s goals match the goals in Denmark.

As they see it, the EU and the national level prioritise the same themes within the educa-

tional field.

Also project managers and managers of organisations involved in LLP consider the LLP

sectoral programmes to interact well with the Danish government’s priorities. For instance,

project managers within Comenius and Leonardo highlight the fact that LLP supports the

current Danish priority to have 95 % of the cohort complete a youth education. A project

manager working with Leonardo projects says:

“It is these kinds of cultural experiences that motivate wavering students.”

A project manager working with Grundtvig projects does not disagree, but emphasises that

the overall LLP (Grundtvig) educational purpose is and should be broader than the current

highly labour-market-oriented educational focus in Denmark.

2.1.2 Added value for target groups

A vast majority of project managers assess that the target group either to a great extent (60

%) or to some extent (36 %) have benefited professionally from the activities (table 16 in the

Annex Report). Likewise, also project participants

indicate in the survey that they either to

a great extent (66 %) or to some extent (28 %) have benefited professionally from the activi-

ties (table 22 in the Annex Report).

More specifically, Table 2 below provides an overview of different aspects in which the

project managers’ have assessed the participants’ benefits from participating in the activities.

The results show that especially ‘intercultural competence’ is emphasised as the aspect in

which the target group to a great extent have improved their qualifications.

Also personal maturity/development, communication, language, cooperation and networking

are to a great extent seen as qualifications improved by the LLP activities. Only conflict

handling is assessed to be improved only to a low extent.

The survey included only project participants from Comenius/Grundtvig Training and Comenius

Assistantship

NIRAS

PDF to HTML - Convert PDF files to HTML files

Table 2: To which extent do you find that the young people have improved their qualifications within the follow-

ing areas through the Programme?

To a great

extent

Language

Communication

Intercultural competence

Conflict handling

Cooperation

Network formation

Personal maturity/development

50 %

51 %

68 %

46 %

43 %

54 %

To some

extent

37 %

40 %

26 %

32 %

43 %

37 %

31 %

To a low

extent

26 %

11 %

Not at all

Do not know /

N/A

27 %

Total

100 %

Project participants themselves also stress intercultural competences to be the qualification

mostly improved, followed by language and communication skills (table 23 in the Annex

Report).

Furthermore, 92 % of the project managers assess that the target group either to a great ex-

tent (52 %) or to some extent (40 %) will benefit from the experience in their future educa-

tion and/or profession.

The picture is similar across the various sectoral programmes; however project managers

within Erasmus, Study Visits and Leonardo are most convinced that the participants will

benefit from their experiences in their future career.

The survey among project participants shows that the target group itself is even more con-

vinced that they will benefit from the experience in their future education and/or profession

(Table 24 in the Annex Report).

Finally, 62 % of project managers and 49 % of project participants assess that the activities

to a great extent have added a European dimension to the target group’s development and

education (Tables 18 and 24 in the Annex Report).

The quantitative survey thus shows that both project managers and project participants as-

sess that the LLP activities to a great extent improve the target group’s professional and

personal qualifications. This indicates a high relevance of the programme.

Most of the target group (70 %) has had aspects of mobility, such as study, traineeship or

exchange visit included in the activities. These mobility activities are by project managers

and project participants assessed to have the appropriate length; 90 % of project managers

and 86 % of project participants are of that opinion. Only 7 % of project managers and 10 %

of project participants deem the Mobility activity to be too short.

The vast majority of the project managers are also satisfied with the content and outcome of

the mobility activities. All in all, 95 % of project managers are satisfied with the target

group’s LLP activities – either to a great extent (77 %) or to some extent (18 %).

Further analysis shows high satisfaction across all sectoral programmes. However, project

managers within Grundtvig and Comenius are even more satisfied with the activities than

project managers within the other sectoral programmes. 16 % of project managers within

Study Visits are either not at all satisfied (5 %) or only to a low extent satisfied (11 %).

NIRAS

PDF to HTML - Convert PDF files to HTML files

The assessment of the Europass Mobility certification is rather varied. 44 % of project man-

agers find that the certification is applicable as documentation for a traineeship or educa-

tional visit abroad either to a great extent (23 %) or to some extent (21 %). 16 % only to a

limited extent (9 %) or not at all (7%) find the certification applicable. A fairly large share of

project managers, however, have no specific knowledge/opinion on this aspect (they answer

‘Do not know / NA’ to the question) (Table 20 in the Annex Report).

Also, 50 % of project participants have no knowledge/opinion of the Europass Mobility cer-

tificate (Table 24 in the Annex Report).

2.1.3 Added value for organisations

One important aspect to encounter when assessing the relevance of the Programme is to

which extent project managers believe the activities to be relevant to their own organisation

or institution. Here, the study shows that a vast majority of project managers (95 %) judge

the activities for which support can be applied through the LLP to be relevant in respect of

the requirements of their organisation/association. This either to a great extent (74 %) or to

some extent (21 %) (Table 14 in the Annex Report).

Furthermore, 92 % of project managers assess that their experience from the project either to

a great extent (56 %) or to some extent (36 %) has a

positive impact

on their own institu-

tion/organisation. Only 7 % judge their LLP experience not at all to have a positive impact

(2 %) or only to a low extent (5 %) to have a positive impact. (Table 27 in the Annex Re-

port)

Regarding the specific impact of the LLP on organisations, the survey illustrates that the

programme in particular has a motivational and promotional effect on the development and

implementation of new ideas and projects. Furthermore, the programme has a positive im-

pact on the professional environment and the study environment, and finally, it improves the

profile and image of the institution/organisation at a national level (Table 31 in the Annex

Report).

Another aspect of the organisational impact of the LLP relates to international cooperation

between partners. The survey shows that most project managers experience that project par-

ticipation indeed contributes to an exchange of experience with other institutions or organi-

sations across borders.

Exchange of experience with

work methods

(e.g. educational materials, educational theory

and practice, etc.) is seen as the aspect which to the greatest extent has been influenced by

participation in the programme. However, also a significant number of project managers

have experienced that their project activities have resulted in

development of education

modules or courses across borders

in the institution or organisation (Table 28 in the Annex

Report).

A majority also assess that the programme to a great extent (47 %) or to some extent (42 %)

has added a European dimension to the institution/organisation (Table 31 in the Annex Re-

port).

Furthermore, eight out of ten project managers believe that the programme either to a great

extent (44 %) or to some extent (36 %) has contributed to the establishment of new partner-

ships with other institutions/organisations. This is the general picture across sectoral pro-

grammes; however,

especially

project managers within Erasmus and Leonardo have this

conviction.

Almost as many, 76 %, assess that the programme either to a great extent (33 %) or to some

NIRAS

PDF to HTML - Convert PDF files to HTML files

extent (43 %) has improved the profile and image of the institution/organisation at an inter-

national level (Table 31 in the Annex Report). Again, project managers within Erasmus and

Leonardo judge the ‘image effect’ to be even greater than project managers within the other

sectoral programmes.

Thus, the results indicate that the Programme has a positive international outcome by in-

creasing cooperation with other institutions and adding international dimensions and per-

spectives to the organisations/institutions.

Furthermore, the study shows that in general it is possible for project managers to adapt LLP

project activities to other activities of the institution/organisation (Table 29 in the Annex

Report).

2.1.4 Long-term effects

The overall relevance of the LLP is closely connected to the long-term effects of the pro-

gramme. The survey indeed shows that project managers in general judge the LLP to pro-

vide long-term positive effects on the institutions/organisations.

Table 3 below illustrates first of all, that 92 % of project managers believe that their project

participation either to a great extent (53 %) or to some extent (39 %) will encourage in-

creased participation in other international activities/projects

under the EU educational pro-

grammes.

And almost as many believe that it will encourage increased participation in other

international activities/projects

in general.

Long-terms effects in terms of increased partici-

pation in

national

activities and projects seem to be slightly less common.

Further, 88 % of project managers believe that their LLP experience either to a great extent

(59 %) or to some extent (30 %) will create

permanent cooperation relations

with persons in

other countries which would otherwise not exist.

Table 3: To which extent do you believe that the participation of the institution/organisation in the project in

which you have participated …

To a

large

extent

… will have a long-term positive effect on the institu-

tion/organisation? (n = 274)

… will create permanent cooperation relations with persons in

other countries which would otherwise not exist? (n = 273)

… to a higher degree than before will mean that Europe and the

European cooperation will be incorporated in the work of the

institution/organisation?

(n = 272)

… will encourage increased participation in national activi-

ties/projects? (n = 273)

… will encourage increased participation in other international

activities/projects in general? (n = 272)

… will encourage increased participation in other international

activities/projects under the EU educational programmes? (n =

272)

24%

To some

extent

To a low

extent

Not at all

Do not

know/

N/A

10%

Total

47%

14%

100%

23%

40%

21%

10%

100%

21%

49%

14%

11%

100%

23%

41%

17%

15%

100%

36%

46%

100%

43%

100%

NIRAS

PDF to HTML - Convert PDF files to HTML files

All in all, the quantitative survey among project managers and project participants indicates

significant added values of the LLP Programme. However, in order to gain the most from

project activities also in the future it is relevant to discern the key lessons learned among

project managers.

When asked about which factors were most important with a view to claiming the greatest

benefit from the programme, the project managers address the need of experience. Particu-

larly, they address the need of experience to fill in the application forms (knowledge of the

specific terminology etc.) but also experience in managing international projects and part-

nerships.

Further, the importance of having continuance in the staff handling the projects is underlined

as well as the importance of staff driven by determination and inclination.

Some organisations make great efforts to improve their foundation for managing interna-

tional projects. Below an example;

”We have pointed out a number of ‘globalisation agents’ at each college. We train them in

guidance and at being inspirational. Some places that works really well.”

(Manager of or-

ganisation involved in LLP, Leonardo)

Summing up, the long-term effects seem especially pronounced in relation to the potential

participation in further EU projects and in relation to the creation of permanent cooperation

relations. Particularly, the benefits and long-term effects tend to appear in organisations that

are experienced in terms of applications and characterized by staff continuance.

2.2

Effectiveness

The below sections describe the effectiveness of the LLP, more specifically the effect of the

integration of prior programmes into one LLP, the implementation of the programme (in-

cluding assessment of administrative procedures), experiences with reaching the target

groups, measures to disseminate LLP results and finally LLP visibility within the educa-

tional community.

2.2.1 Integration of previous programmes into one LLP

The IU staff agrees that it is a good thing that sectoral programmes have been integrated

within the LLP. It means more cooperation and the opportunity to relocate funds across the

different sectoral programmes in order to ensure better budget absorption.

Furthermore, some judge the integration to have a positive effect on the activity level within

the sectoral programmes (especially Erasmus). However, some IU representatives under-

score that adequate funds have not followed the increased activity level (especially within

Grundtvig).

Most project managers (based on the qualitative interviews) say that they have not noticed

the merger and/or that they have not felt a difference at all. That is evident for the specific

project managers within Erasmus, Comenius, Leonardo and Grundtvig. Only a few project

managers have noticed a change. One of them says:

NIRAS

PDF to HTML - Convert PDF files to HTML files

“The application processes and administrative work in general are faster now…At least for

Grundtvig.”

(Project manager, Grundtvig)

The dialogue meeting gave the impression that beneficiaries across sectoral programmes

would like to see more interaction and have more possibilities to bridge the sectoral pro-

grammes in the future.

2.2.2 Information and consultation procedures

Most project managers have their knowledge about the programme from the IU – the Danish

Agency for International Education (64 %). Many also have their knowledge from previous

experience with applications (49 %) or from more informal sources of information such as

colleagues (33 %) and friends/network (16 %). The least relevant source of information is

the EU Commission (9 %) and the media, including specialist journals (3 %).

Thus, the IU is the most important source of information about the Programme and a vast

majority of project managers (94 %) have indeed used their website, www.iu.dk. Most of the

users are furthermore satisfied with the usability of the website – 95 % are either very satis-

fied (31 %) or satisfied (64 %) with the website. Only 3 % are dissatisfied.

The IU website is, however, not the only source of information that the IU provides. Besides

personal support, the IU offers project managers a number of services, e.g. information

meetings, information material/brochures, electronic newsletters, and the

www.udiverden.dk

and

www.skoleniverden.dk

homepages.

The survey shows that not all of the services are used by project managers. For instance,

more than half of the project managers are not familiar with the two websites. Instead most

project managers have their knowledge of the programme from written material (brochures

etc.) or from participation in meetings and/or because they receive an electronic newsletter

from the Agency.

Of the various information services that the IU provides, the information meetings are as-

sessed most positively followed by the written material (electronic newsletter and bro-

chures). Those who use the two websites also announce a fair amount of satisfaction (Table

6 in the Annex Report).

In general, the evaluation shows that project managers are very satisfied with the personal

support and guidance provided by the IU. It concerns the application process as well as the

completion of the activities.

“They have been excellent and helpful all through the project. Five out of five stars.”

(Man-

ager of organisation involved in LLP about the IU service level)

2.2.3 Application and reporting procedures

Two thirds of the project managers used an electronic application form and one third used a

paper-based application form (Table 8 in the Annex Report).

71 % of the project managers received support from the IU in connection with the applica-

tion process and the implementation of the activities. Of those receiving support there is a

great amount of satisfaction with the support given, as mentioned above.

All together there is a fairly high level of satisfaction with the application process. 27 % are

very satisfied and 66 % are satisfied. 7 % are dissatisfied with the process. This might be due

NIRAS

PDF to HTML - Convert PDF files to HTML files

to the support and guidance available in relation to the application process. Almost three out

of four are very satisfied with the IU support in this respect.

The same pattern is apparent regarding the support in connection with the

implementation

the activities; 62 % are very satisfied, 23 % are satisfied and only 1 % are dissatisfied

(Ta-

ble 10 in the Annex Report).

Table 4 provides an overview of the level of satisfaction among the project managers regard-

ing the various administrative procedures. Compared to the general level of satisfaction with

the application process the table gives a more varied picture. For example, a relatively high

percentage –22 % – are dissatisfied with the time span between the deadline for the applica-

tion and the time of approval. Further analysis documents that this specific dissatisfaction is

most pronounced for Comenius project managers. Among these 39 % are either dissatisfied

(31 %) or very dissatisfied (8 %) with the time span between the deadline for the application

and the time of approval. This could be due to the fact that pre-schools, primary and secon-

dary schools (the Comenius target groups) traditionally plan for the coming school year

rather early and therefore also have a more urgent need to know if their application is ap-

proved or if other activities should be planned instead.

Also, the application form in itself is somewhat criticised as 13 % announce their dissatisfac-

tion with it. The level of satisfaction with these two aspects is, however, still substantial,

ranging from 72 % satisfied (‘very satisfied’ and ‘satisfied’ combined) to 84 % satisfied.

Table 4: How satisfied have you been with the administrative procedure in connection with the following?

Very

satisfied

The application form (n = 330)

The time span between the deadline for applications and ap-

proval (n = 330)

Final reporting of exchange/project activities (n = 329)

Requirements of the financial reporting

(n = 329)

Requirements of the reporting as regards contents (n = 329)

Payment of the grant (n = 328)

20%

13%

22%

17%

32%

Satisfied

Dissatis-

fied

13%

22%

Very

dissatis-

fied

Do not

know /

N/A

18%

10%

Total

64%

59%

60%

57%

63%

56%

100%

The survey shows a high level of satisfaction with administrative procedures in connection

with financial issues. 88 % are either very satisfied (32 %) or satisfied (56%) with the ad-

ministrative procedure in connection with the payment of the grant. 79 % are either very

satisfied (22 %) or satisfied (57 %) with the requirements of the financial reporting.

In general, project managers with prior experience with EU educational programmes explain

that they experience the administrative procedures to have become easier, more flexible and

less bureaucratic during the last years.

Further, an important indicator of the assessment of the level of administrative burdens is the

degree to which the project managers think that the resources invested in the administrative

procedures have been worth while, which seems to be the case. 79 % of project managers

14 % answer ’Do not know / not applicable’ to the question.

NIRAS

PDF to HTML - Convert PDF files to HTML files

assess this to be the case either to a large extent (23 %) or to some extent (56 %). However,

one out of five project managers believe that the administrative workload only to a low ex-

tent (16 %) or ‘not at all’ (4 %) is reasonable when looking at the level of the grant given

(Table 44 in the Annex Report).

On the basis of the above results it therefore seems safe to conclude that the administrative

procedures are relatively well constructed in that there is an overall satisfaction with the

procedures. However, the results also indicate that there is still a huge potential for im-

provements in terms of facilitating less bureaucratic and more flexible administrative proce-

dures.

In the survey, project managers were asked to indicate up to three factors which have im-

peded the completion of project activities. The most dominant factors in this respect are lack

of recourses (35 %), the administrative requirements (31 %), lack of support and/or interest

from colleagues (28 %), lack of interest from the target group (15 %) and lack of support

from the faculty or the management (11 %) (Table 46 in the Annex Report).

Thus, even though the relevance and added value of the programme is assessed to be posi-

tive, the above factors seem to make the implementation more difficult. Only few (10 %) list

the lack of guidance and even fewer list the lack of interesting possibilities as obstacles to

the implementation of project activities.

2.2.4 Recruitment of participants

Central to the effectiveness of the Lifelong Learning Programmes is the ability to recruit

participants. The survey shows that 38 % of project managers either to a great extent (4 %)

or to some extent (34 %) faced problems in the recruiting process (Table 41 in the Annex

Report).

Further analysis highlights significant differences between sectoral programmes in relation

to recruitment difficulties. Project managers within Erasmus experience the highest degree

of recruitment difficulties (17 % to a high extent and 52 % to some extent), followed by

project managers within Grundtvig (no one to a high extent but 52 % to some extent), and

Leonardo (3 % to a high extent and 40 % to some extent). Within the Comenius sectoral

programme one third of project managers experience problems either to a high extent (3 %)

or to some extent (33 %) with the recruitment of participants.

On the contrary, only few project managers within LLP Study Visit experience recruitment

difficulties (14% to some extent) (Table 41a in the Annex Report).

The qualitative studies highlight an important aspect of the recruitment difficulties in rela-

tion to teachers in specific. Thus, many explain that it requires some extra planning to cover

the lost schedules of teachers going abroad. Furthermore, some stress that it can be hard

motivating the teachers and getting them to sacrifice their weekends voluntarily.

The survey does not specifically address the difficulties of recruiting young people, but the

interviews point to the fact that many students are very ‘established’ at a young age. They

have partners, flats, jobs outside school, and this makes it hard for them to choose to go

abroad for three weeks, or a semester.

In the case of Leonardo, there seems to be another obstacle, and that is the employers of the

young people, who do not always see the benefit of an international experience. For in-

stance, when it comes to the trades, the standards and methods differ, and employees have

not always realised that internationalisation is affecting them too, according to some of the

interviewed project managers etc.

NIRAS

PDF to HTML - Convert PDF files to HTML files

“The difficult part is to get the employees, for instance an electrician, to realize that it is

relevant [read: to go abroad]. They seem to think that learning languages is not important.”

(Manager at organisation involved in LLP, Leonardo)

2.2.5 Participant profiles

A total of 61 % of LLP participants are pupils, students or course participants, and they can

be all ages. 27 % are teachers who participate in training or competence development. They

are all usually providers of knowledge but through the programme they get the opportunity

to learn more (Table 39 in the Annex Report).

Table 5 indicates a fair level of geographical representativeness among the participants in

the Lifelong Learning Programme. Compared to the proportions of the population living in

the different regions of Denmark the participants’ distribution seems very representative.

Approximately a quarter of the participants live in Southern Denmark. Northern Jutland,

which is the most sparsely populated area, has the fewest participants with 12 %. Copenha-

gen – the capital of Denmark –surprisingly has only 19 % of the participants.

Table 5: From which region do the participants primarily come?

Percent of participants

Copenhagen

Zealand

Southern Denmark

Central Jutland

Northern Jutland

Do not know / the project has not in-

cluded Danish participants

Note: n=311

19%

15%

27%

23%

12%

Percent in population

30%

15%

22%

23%

11%

The table shows that the effectiveness of reaching the target groups geographically is ac-

complished to a high degree. There are participants of all regions of Denmark in the Life-

long Learning Programme and the participation is fairly representative compared with the

distribution of the population in the regions in general.

Table 6 reports that only few participants have special needs. Very few (3 % and 5 %) have

physical or mental disabilities. The most common special needs are needs related to school

education such as reading difficulties, language difficulties and learning disabilities. 10-15

% of the participants have these special needs.

The table thus shows that the programme to a lesser extent reaches the target group with

special needs.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Table 6: Which special needs have there been?

Percent

Reading difficulties

Language difficulties

Learning disabilities

Mental disability

Physical disability

Other special needs

Do not know / the project has not included participants with

special needs

Note: n = 311

15%

13%

10%

Even though participants with special needs are not dominant in the projects, IU representa-

tives and project managers (based on the qualitative interviews) seem to be very aware of the

horizontal policies, that is, the LLP's overriding purpose to increase cultural and linguistic

diversity, counteract racism etc.

Project managers have incorporated the goals in their projects and all agree that it has been

implemented very successfully. For example, some highlight their efforts to help those stu-

dents who have few financial means, others emphasise that the target groups of their projects

specifically include children with special needs, people with other ethnic backgrounds than

Danish, etc. Finally, some also mention that only organisations specifically oriented towards

the horizontal policies are considered partners in these EU projects.

Summing up, when it comes to participant profiles the evaluation shows that there is a fair

level of geographical representativeness, whereas the representation of persons with special

needs seems more limited. However, the analysis also shows that the horizontal policies in

relation to cultural and linguistic diversity etc. are supported via the nature of the specific

projects.

2.2.6 Dissemination of LLP results

Based on the qualitative interviews it seems that the dissemination and exploitation of LLP

activities is a point of division for project managers. Some (project managers within Leo-

nardo, Grundtvig and Comenius) explain that they have evaluation forms, write letters dur-

ing travelling, produce brochures when returning, write articles on the school website and

require all participants to give a verbal presentation of their experiences to other students or

pupils.

“Students, who have been on trips, are there to tell the new students about their experiences.

We have only heard positive stories.”

(Manager of organisation involved in LLP, talking

about Leonardo and Comenius)

Others tell a different story. For instance, project managers within Erasmus seem to work

less goal-oriented with the dissemination of LLP results and student experiences. Thus, they

tend to agree that they could benefit from more activities within this area and explain that

they are working on that.

NIRAS

PDF to HTML - Convert PDF files to HTML files

The National Agency in Denmark also contributes to the focus on dissemination and exploi-

tation. The IU staff explains how the dissemination and exploitation has a much stronger

focus in the applications for the LLP Programme than in previous programmes and how it is

a recurrent theme when project managers are brought together for training or information

purposes.

The IU has used both their website and written material to disseminate results and has pro-

duced a number of publications to that end. An interesting example is the book on creativity

and innovation, produced in collaboration with the Youth in Action Programme, presenting

ten projects focusing on creativity and innovation. The book is the output from a Conference

held in relation to the European Year of Creativity and Innovation in 2009. Each year a the-

matic conference has been held and publications have been drawn up subsequently.

This has proved a very good way of getting the message about the LLP – and the YiA , as

well as results from projects – out to a larger audience, and it could be followed up by other

initiatives that bridge the programme.

The dissemination channels used today are relevant, but more could be done, if the neces-

sary resources were available. For instance, the qualitative interviews suggest ways to use

the web more actively, with more reports from projects, frequently asked questions regard-

ing dissemination and exploitation and good advice from project managers to other project

managers on these issues. And, finally, the dissemination of the programme and the results

of the projects could be accentuated in more cross-sectorial events such as the dialogue

meeting arranged in connection with the evaluation.

2.2.7 Visibility of the LLP in the education and training community

Based on the qualitative interview, the awareness of LLP seems to be quite high among po-

tential target groups. Generally, the IU staff and most project managers agree that most edu-

cational institutions know about the LLP Programmes; however maybe with the exception of

very new target groups.

The fact that the name – LLP – is new, and the ‘old’ names Leonardo, Comenius etc. still

exist, create an interesting schism. On the one hand LLP is perhaps not very well established

as a brand, while the sectoral programmes are. From both the IU and the beneficiaries this

calls for a suggestion to not change the names of the sectoral programmes again, because it

takes time to incorporate a new name in the mind of the target group.

Several project managers point to the fact that there seems to be a potential in addressing the

student level more directly with LLP marketing, to attempt to create a further pull factor.

2.3

Efficiency

This section describes the efficiency of the LLP, more specifically the cooperation between

authorities, the management supporting tools and the level of financial support.

2.3.1 Cooperation between authorities

The Danish Agency for International Education implements the LLP Programme under the

authority of the Ministry of Science, Technology and Innovation. Acting as NA the IU pro-

vides information about the programme, sectoral programmes and specific actions of the

NIRAS

PDF to HTML - Convert PDF files to HTML files

decentralised parts of the programme, gives advice to potential applicants, administers the

grants and controls and financially manages the granted projects.

This structure seems logical and clear and works well to enable a successful implementation

of the LLP Programme in Denmark. Regarding the division of the programme into decen-

tralised and centralised actions it seems to be working well enough, especially for those who

are well informed and know the difference. The decision taken by the European Commission

and the Executive Agency that NA’s are not entitled to provide counselling on centralised

activities is however creating some frustrations within the NA. They feel that they let the

applicants down when they have to turn them away to the Executive Agency for advice on

the centralised activities, because they know that this does not function as well as it could for

Danish applicants.

“The division in central and de-central doesn’t work well for us. We cannot advise users

properly on that area.”

(NA staff, focus group)

The workload and the control have increased seen from the IU perspective, both nationally

and centrally. But there is a general understanding that control is necessary. The Ministry of

Science, Technology and Innovation is responsible for the monitoring and control procedure.

This ministry however cooperates with the Ministry of Education who is responsible for a

number of the educational areas included by the LLP. The IU staff expresses their general

satisfaction with the way in which the cooperation and the coordination between the two

ministries work.

From the interview with the staff of the Agency the impression is, furthermore, that the co-

operation with the Commission has been good throughout the period, and that the communi-

cation and coordination structures set up for the programme work well.

However, a major criticism regards the reporting requirements, especially the financial re-

porting, from the NA to the Commission, which seemingly continues to grow. The IU staff

address the shift from earlier having to report more ‘contents’ to now having to report more

financial aspects. They also stress the point that they now have to report on a lot more pa-

rameters and that their narrative descriptions have increased in extent. The fact that all the

data collected by the NA does not seem to be used in any visible way adds to the dissatisfac-

tion.

Seen in the light of the fact that the LLP is now one programme the staff also seem to ex-

perience a large variety in the way that the reporting requirements are handled centrally,

which does not help the streamlining of the processes of sectoral programmes in Denmark.

The IU staff expresses that the workload has grown in the first years of the LLP compared to

the former programme but tentatively explains this as the adjustment to new forms, new

actions etc. both internally and for the participants.

The system of divided responsibilities has been implemented. It is seen as a natural conse-

quence of the enhanced control by the Commission, and the system is upheld, although it has

some practical consequences and strains on the staff for a relatively small NA like the Dan-

ish, when the same person cannot advice on and control the same project.

NIRAS

PDF to HTML - Convert PDF files to HTML files

2.3.2 Management supporting tools

There is a number of different management supporting tools, e.g. LLP Link, EST, ADAM

and Circa (replacing NETY), developed in order to facilitate an efficient management and

implementation of the programme.

Among the people responsible for the implementation of the LLP at the IU there seems to be

modest satisfaction with the new administrative tool LLP Link. The main gain seems to be

that the EC can pull data from it directly, but as a national supporting tool it is not consid-

ered to be very user friendly.

The IU representatives explain that they have to do less typing with the introduction of LLP

Link. However, they experience that the technological requirements to operate the systems

have increased. This calls for more training, but also better technical support, both nationally

and on a European level.

IU representatives furthermore stress the point that even though the new administrative sys-

tem might be easier to operate, it is not seen as a relief for users (project managers). They

still have to mail in application forms in paper as well as fill out the electronic form, which

confuses them. There seems to be still room for improvement in the development of systems

to handle applications and reporting using modern technologies.

The system Circa, is considered to be helpful. An IU representative explains: ”You

get the

idea of it all. Once we got the hang of the system, it’s really smart.”

2.3.3 The level of financial support

Among project managers there is a high degree of satisfaction with the level of financial

support. 37 % assess that the level of financial support has been adequate to a great extent in

respect of meeting the original objectives of the activities, and 54 % assess this to be the

case to some extent. Only 8 % believe that the amount of financial support has been ade-

quate to a low extent and 2 % assess that the grant is ‘not at all’ adequate.

The qualitative interviews with project managers support this information. The project man-

agers generally find that the students/participants receive reasonable financial support – in

particular as regards travel expenses. However, they believe that in some cases more money

could be allocated for administrative support to the coordinating partner, and for instance to

the hosting partner in order to make it possible for hosts to be paid for their extra time.

Furthermore, the survey illustrates that only few projects are not totally dependent on the

level of support from the EU. Table 7 documents that, according to project managers, almost

two in three projects could not have been executed without EU funding, and only 9 % of the

project managers find it likely that their projects would be carried out without financial sup-

port from the EU.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Table 7: To which extent do you find it likely that the activities completed in the project

would have been completed, had there been no grant schemes under EU educational

programmes?

Percent

To a great extent

To some extent

To a low extent

Not at all

Total

Note: n = 305

28%

64%

100%

On average, 14 % of project managers have received financial support for the activities from

elsewhere that the EU (table 12 in the Annex Report). Behind this number, though, lie sig-

nificant differences between sectoral programmes, as 34 % of Erasmus project managers

have received other kinds of support against only 8-12 % of other project managers (Table

12a in the Annex Report).

However, most of the project managers who have received support also from elsewhere

judge that the assurance of financial support from the EU has been an advantage in this re-

spect (Table 13 in the Annex Report). Thus, the award of Community funds serves to some

extent as a catalyst for obtaining other financing.

NIRAS

PDF to HTML - Convert PDF files to HTML files

Conclusions and suggestions for

future developments

3.1

Conclusions

This chapter sums up the main conclusions of the interim evaluation.

Based on the survey and the interviews and the analysis of the statistical data available, the

LLP Programme implementation 2007-2009 is assessed to be relevant as well as quite effec-

tive and efficient.

The LLP Programme is considered to be complementary to Danish national activities and

policies, and the evaluation points towards significant added value for target groups as well

as organisations. At target group level the LLP activities to a great extent improve the pro-

fessional and personal qualifications of project participants. At organisational level the

evaluation shows that the programme to a high extent has added a European dimension to

the institution/organisation in terms of enduring international partnerships in many cases.

The study outlines a number of important factors (lessons learned) in order to claim maxi-

mum benefit from the programme. Particularly, project managers address the importance of

experience, that is, experience to fill out the forms of application as well as experience in

managing international projects and partnerships. Further, the importance of having continu-

ance in the staff handling the projects is underlined as well as the importance of staff driven

by determination and good will.

This interim evaluation shows some important changes from the previous programmes:

Firstly, the recommendation to ensure more organisational rooting of projects seems to have

been met in the new projects under LLP. Based on the interviews there is evidence of LLP

projects being more a part of a strategic focus, an internationalisation strategy, than previ-

ously. Secondly it seems as if dissemination has grown into an integrated part of the project

‘make-up’, and project managers are very focused on how to disseminate – and the impor-

tance of disseminating – and this is also a clear improvement from the previous programme.

Efforts among project participants to disseminate programme experiences seem to be quite

diverse. In general, there seems to be a potential connected to a more coordinated approach

among projects. At national level there also seems to be a potential for further active use of

the web, with more examples of projects, frequently asked questions and good advice from

project managers to other project managers. And, finally, the dissemination of the pro-

gramme and the results of the projects could be accentuated in more cross-sectorial events

such as the dialogue meeting arranged in connection with the evaluation.

Despite quite a high level of satisfaction among project managers with administrative proce-

dures in general there seems to be a clear potential for improvement in some respects.

Mainly the time span between the deadline for the application and the time of approval is

stressed as problematic. The results also indicate that there is still a huge potential for im-

provements in terms of facilitating less bureaucratic and more flexible administrative proce-

dures.

NIRAS

PDF to HTML - Convert PDF files to HTML files

The focus on financial aspects in the reporting tends to shift the focus from the contents and

the real value of the projects to more quantitative aspects, and this counteracts the emphasis

on dissemination of results, exploitation and transfer of knowledge, innovation and experi-

ences, that runs parallel in the LLP Programme.

Also in relation to the recruitment process there seems to be room for improvement. Almost

four out of ten project managers to some extent face problems in the recruiting process. Es-

pecially regarding teacher mobility there tends to be some barriers that have not yet been

overcome

The survey illustrates that only few projects are not totally dependent on the level of support

from the EU. According to project managers, almost two in three projects could not have

been executed without EU funding. Finally, the award of Community funds serves to some

extent as a catalyst for obtaining other financing.

3.2

Suggestions for LLP 2010-2013

Based on the current evaluation results the evaluators suggest the following points for LLP

2010-2013, and for future programmes.

3.2.1 Flexibility and synergies

The integration of the previous educational programmes into one LLP Programme launched

in 2007 was a political statement from the EU Commission rather than a wish from the edu-

cational sectors across Europe. Traditionally the sectors have lived their separate lives, been

under separate laws and regulations, and their interfaces have been limited. The LLP Pro-

gramme was designed to create more interfaces – more interaction and more synergy be-

tween them, in order to, as it is stated in the decision of the Commission in 2006

“contribute

through lifelong learning to the development of the European Union as an advanced knowl-

edge society, with sustainable economic development, more and better jobs and greater so-

cial cohesion”.

The integration of the sectoral programmes into one LLP is judged positively by IU staff,

because, as mentioned above, it enables them to cooperate more and gives the opportunity to

relocate funds across the different actions and sectoral programmes. There is no doubt that

the LLP Programme has meant some changes in their way of working. However, the survey

shows that interestingly enough, most project managers seem not to have noticed the inte-

gration and/or have not felt any difference. And what is more important, the interviews and

the dialogue meeting show that they see a much larger potential in cooperation across pro-

grammes than what is happening today. Therefore, the evaluators conclude that the full po-

tential in terms of maximum synergies has not yet been realised. In spite of improved com-

munication and coordination across sectoral programmes, these still tend to live quite sepa-

rate lives. This is not least apparent from the very dissimilar and often incomplete statistics

provided by each sectoral programme (as presented in the Annex Report).

“If the boundaries were fluent it would be an advantage (for the operation). There is a joint

office, but it is different people administrating. It’s not actually merged.”

(Project manager,

dialogue meeting)

The evaluators recommend that a further effort be made to

ensure and realise the synergies

of the LLP Programme –

internally in the NA, but equally important towards the benefici-

COM DECISION No 1720/2006/EC

NIRAS

PDF to HTML - Convert PDF files to HTML files

aries who should be supported and inspired to find ways of combining activities also across

to the Youth in Action Programme or centralised measures, where applicable. In order to

visualise the synergies to the participants the IU could provide thematic seminars across

sectoral programmes, or in other ways bring people together to share experiences. The dia-

logue meeting in connection with this interim evaluation was a good example of how fruitful

discussions could take place across sectors.

The evaluators further stress that there is a need for a general overarching LLP focus on the

beneficiaries and the overall project idea more than what box to tick in the application. More

concretely, the wall between sectoral programmes and actions must be torn down in order to

meet the overall objective of the LLP. This runs as a strong undercurrent in the qualitative

interviews and in the dialogue meeting. Other restraints such as length of stays, participants

with other nationalities, participants on public support etc. are also seen as irritants – and

should, if possible be eradicated to enable the free flow of participants in lifelong learning.

3.2.2 Administrative cohesion

The bureaucracy surrounding the LLP Programme is evaluated in the report. The main con-

clusion to that is that the project managers are not very unhappy with the application proc-

ess. It should be noted however, that this is a study among those who had their projects

granted, and that there is a rather high satisfaction – three out of four are very satisfied with

the support. Though, when investigating the qualitative responses and the comments added

to the questionnaire, a different picture emerges.

The bureaucratic requirements are seen as a great – but apparently not insurmountable –

barrier to the LLP Programme, and many requests are made to remedy this: ensuring that

application forms are not changed repeatedly and that reporting requirements are available at

the time of the launch of the programme, more wide-spread use of new technologies, less

repeating of factual information etc.

Another aspect concerning the administrative burden of LLP is the fact that the applications

and reporting forms are filled with EU words. The survey and interviews suggest that for

some project managers – especially those who do not have a long experience in the area to

draw from – the EU terminology is in danger of becoming empty and disconnected from the

reality of their project, Consequently the evaluators stress that it is important to work with

this constantly to secure the relevance – and the understanding of relevance across the whole

programme.

Many of these things have to be solved at an EU level, but there are also aspects that could

be addressed nationally.

The evaluators recommend that further effort be made at EU level to

decrease the adminis-

trative burden

on projects. The control is necessary, and this is not a question of abolishing

all control measures, but a constant focus has to be on the questions asked and the require-

ments put up to be eligible for LLP funding. It should be clearer what all the information is

used for at a European level.

At national level the evaluators would like to stress that it is important that the IU staff con-

tinues the good work and are given the resources to keep this up. As is shown in the evalua-

tion the support of the IU staff is seen as indispensable to both new and experienced appli-

cants, and this cannot be underestimated. The evaluators recommend that the possibility of

obtaining guidance for potential applicants is made even more visible, so that it does not

exclude anyone. Furthermore, the evaluators recommend that the IU could work even more

with the website in order to give more good examples of project ideas, applications, dis-

semination activities, types of cooperation, partners, FAQ etc.

NIRAS

PDF to HTML - Convert PDF files to HTML files

3.2.3 Systemic impact

The link to the national priorities for the LLP Programme is evident: internationalisation,

formal education, development of competencies, lifelong learning. There are also examples

of how these aspects, especially the internationalisation, have gained a foothold in the insti-

tutions involved in the LLP, and there is definitely an improvement on this aspect as com-

pared to the previous programmes, but there is still room for improvement.

On the basis of the evaluation the evaluators therefore recommend that the LLP further

stresses the requirements for project design and project implementation to give the pro-

gramme better chances of

realising a systemic impact

on the educational system in Den-

mark. There should be more focus on involving for instance municipalities, regions, and the

Ministry of Education in projects to develop the education of the future. It is also recom-

mended that more systematic use be made of the experiences of mobility, and closer connec-

tion should be established between mobility activities and development projects to promote

the institutional innovation through the involvement of teacher groups, mid-level manage-

ment etc.

3.2.4 Information and dissemination

The IU does a very fine job in

informing about the LLP Programme

in Denmark as

shown above. This is an indispensable activity and one that has to be maintained constantly.

The use of websites is well established, and the survey shows a high level of satisfaction

with these aspects. The evaluators recommend that the information activity for LLP is main-

tained, and the website developed as mentioned above.

The evaluators furthermore recommend that more effort be put into information to reach

those who are not yet involved in the programme. More information to pupils and students

focusing on ways to overcome the barriers to mobility could be one way of helping the or-

ganisations break down these barriers, as well as more broadly based campaigns on interna-

tionalisation, or a road show with young people who tell about their international experience

in schools, or teachers telling about their experience when and where teachers are gathered.

Another way of reaching new actors or new target groups is a more structured development

of areas where there are ‘black holes’ – be that geographically, in terms of organisation

types, etc.

Dissemination of project results and experiences is also very important and in the LLP Pro-

gramme the focus on DEOR (dissemination and exploitation of results) has been growing

with the strengthened emphasis centrally. The focus on dissemination means that participat-

ing organisations/institutions/schools at all levels should allocate means and time to dis-

seminate results internally and externally to spread the experiences obtained throughout the

organisation. The interviews especially show that this is happening to a certain degree, and

the projects experiment with different ways of living up to their dissemination obligations.

The evaluators recommend that the focus on dissemination is kept in the reporting proce-

dures and in the control visits. However, it is important that the focus on dissemination is not

just quantitative, i.e. number of dissemination events, but also with has focus on contents

and methodology. This could be gathered and made available to other projects as a good

practise guide to dissemination. Regarding dissemination the evaluators furthermore rec-

ommend that the Agency should continue the support to dissemination initiatives in the pro-

jects separately, but even more importantly create more joint dissemination channels, like

the book on creativity and innovation produced with YiA or other such initiatives

Further among project managers there is strong wish to exchange experience with other pro-

ject managers, and in that respect the evaluators suggest that the IU consider establishing

NIRAS

PDF to HTML - Convert PDF files to HTML files

networks among project managers in order to provide the possibility to share knowledge and

experience with like-minded people at national level during project implementation, and not

just virtual networks, but more possibilities of meeting, perhaps even regionally.

“I’ve missed having contact with project leaders in other municipalities. I’ve missed coop-

eration or talks with other project leaders. Some knowledge sharing on a Danish level.”

(Project manager, interviews)

3.2.5 Inclusion

The evaluators recommend that there be strengthend

focus on special needs and inclusion

in the LLP Programme It should be considered if central demands in relation to inclusion

factors in projects should be enhanced. Further, it should be considered if the IU could pro-

vide financial assistance to people with only few means and/or personal resources, e.g. sin-

gle mothers, disabled persons, etc.

In this respect the evaluators stress that it is important that the inclusion concept is seen as

broadly as possible, to also include projects or activities in other languages than English, and

to include people outside Europe in projects or activities and make sure that people with

disabilities, blind people or hearing impaired persons can participate in all training activities.

It is also recommended that the Commission thinks inclusivity when planning the new pro-

grammes and their administrative procedures.

3.2.6 Quality and qualifications

Focus on quality and qualification is one area that has been underlying the LLP Programme

as such, but on the basis of the interviews and the dialogue meeting the evaluators recom-

mend that it be given even more focus in the coming years. These issues are, furthermore,

sector-independent and could be used to gain more integration and synergy between sectors.

Teacher mobility – both outgoing and incoming – is another area that according to the

evaluators could be given even more focus as it can be a push factor for increased student

mobility. However, by giving the teachers an international experience they can bring this

back in the classroom and thereby add to the impact on all the pupils/students they teach and

create a major added value. In the light of the fact that the survey reports on some difficul-

ties in recruiting teachers, it should be investigated whether certificates and/or financial in-

centives could be applied in order to motivate teachers, and the IU could be the motor be-

hind finding these good examples in Denmark.

”It is important to have an exchange of teachers: To have teachers from abroad coming to

the school. It would be nice with more of that.”

(Manager of organisation with LLP projects,

Leonardo)

The involvement of companies in projects is also an area where the evaluators see that more

synergy and more relevance could be built up. A large part of the companies are becoming

more and more internationally oriented and they have seen the need for employees with an

international dimension in the education. This could be used to influence those groups of

young people who are still not very convinced. This pull factor could be a way of ensuring

that mobility be spread more evenly and not just considered a concept relevant for students

in higher education institutions.

NIRAS