
For the
Information Society
and
Regional Development
|
LOCREGIS projects have been funded by the European Commission More information available in http://www.locregis.net This handbook is prepared by the LOCREGIS
II project team as a part of the final report of the project for the European
Commission DG REGIO Helsinki September 2000
|
Foreword
One of the main tasks of the two LOCREGIS projects implemented in the years 1996-2000 has been to develop evaluation tools for recognition of the best practice projects. Many hundreds of hours have been used to discuss, which features in the project are best or at least good practice for the regional development and how it should be measured.
The first versions of the LOCREGIS best practice criteria were based on the close co-operation with the project leaders. The results of this work were very fruitful, but sometimes the needs of the public funding were difficult to understand on the grassroots level. E.g., project leaders were thinking that the need of transferability was stealing of the work done.
The latest version, described in this handbook, is the result of LOCREGIS II, where the main architects of the best practice were professionals from the regional government. The viewpoint was different, but anyway the work was based on the first version and so the result can be seen as a compromise between those two aspects.
LOCREGIS tools have prepared to analyse the situation so transparent as possible and not just from the viewpoint of one certain programming document or initiative. Especially on the less favoured regions all possible actors are needed and so the sum of the information society projects is more important that one single project.
Some limitations in the use of the criteria are worth mentioning:
On behalf of the project management group I have the pleasure to express warm thanks to everyone who participated in this innovative work, in working group 1 of the project and especially to the Regional Council of North Karelia for providing the regional benchmarking experiences to the wider use.
Helsinki 28th of September 2000
Heikki Lunnas
Project manager
Content
2.2. Development of the Criteria 1999-2000
Project
leading, planning and guiding
Use
of the background documents
4. LOCREGIS BEST PRACTICE CRITERIA
A.2 Create demand for the use of existing infrastructure and services
A.3 Provide a higher quality of life for the people
A.4 Enhance economic resources
A.5 Increase skilled potential on the area
A.6 Improve image and enhance credibility
B.1 Innovativeness concerning the content and tools on the own region
B.2 Innovative concerning the content and the tools on European or
global level
B.3 Innovative for specific target groups
B.4 Innovative by reengineering the existing structures
B.5 Innovative in the project implementation
B.6 Level of challenges in the implementation
C.1 Involves all natural and/or important partners
C.2 Involves the private sector appropriately
C.3 Supports creation of new regional networking platforms
C.4. Optimises the competencies of the partners
C.5. Project displays transregional features
C.6. Has a potential to continue after the project
D.2. Involvement of the end-users
D.3. Motivation and commitment of the partners
D.4. An ongoing controlling system
D.5. A sustainable development of the results and experiences after the
project
D.6. Dissemination of the results
E. Regional development policy
E.1. Compatibility with the regional development strategy
E.2. Connected to the regional or national clusters
E.4 Local or regional spearhead/key project
E.6. Potential leverage effects
5. LOCREGIS REGIONAL INFORMATION SOCIETY
PROFILES
6 Profiling the Projects in the LOCREGIS
Database
7. Adaptation of the Methodology in North
Karelia
7.1 RISI – Project NOKIS and Project Evaluation
Principles
of the evaluation system
Implementation
of the evaluation system
7.3 Examples of the NOKIS Evaluation
Project
Learning Upper Karelia
The local and regional information society project LOCREGIS II was a follow-up project of LOCREGIS (LOCal and REGional Information Society), a project that was initiated by the European Union and supervised by DG XVI during the years 1996-98. The main target areas of LOCREGIS I were the new European Union members states Austria, Finland and Sweden and especially their Objective 1 and Objective 6 regions. The LOCREGIS projects have been co-ordinated by the Finnish Association of Local and Regional Authorities.
The aim of LOCREGIS I was to prepare an inventory and analysis of information society projects that can strengthen the competitiveness of less favoured European regions through innovative use of information technology.
Among its products, LOCREGIS set up networking platforms, a database of information society projects in the partner countries, and developed a system of best practice criteria for evaluating information society projects from the point of view of regional development.
LOCREGIS became a tool for:
The main results of the LOCREGIS I have been presented in the
address http://www.locregis.net/.
As a follow-up project to LOCREGIS I, LOCREGIS II builds on the results, recommendations, experience and methodology of its predecessor.
The tasks in LOCREGIS II were:
LOCREGIS II project has been coordinated by the Association of the Finnish Local and Regional Authorities and implemented with following partners:
Ø ARCO Belgium
Ø BIC Burgenland and Wirtschaftsservice
Burgenland Austria
Ø County Administrative Board of Jämtland Sweden
Ø DataCity Turku Finland
Ø Eco Plus Austria
Ø Federation of the Swedish County Councils
Ø IT Blekinge Sweden
Ø IT Norrbotten
Ø Municipality and Region of Gotland Sweden
Ø Regional Council of Kainuu Finland
Ø Regional Council of North Karelia Finland
The first version of the criteria was formulated at the end of 1996. The basis of the criteria was the experiences of those people who had participated in the work and also some existing studies and reports on the subject. Already in the early stages of LOCREGIS project information on local and regional information society projects co-financed by the Structural Funds was gathered. In particular, projects implemented in the Objective 1 and 6 regions were of interest to LOCREGIS.
About 20 of the projects were
selected to be examined in more detailed in the first LOCREGIS seminar
organised in Kuusamo, Northern Finland, in the beginning of 1997. Most of these
projects have been essential for the work done in the LOCREGIS project and have
been thus called core projects.
During the spring of 1997, the first version of the best practice criteria was further developed and was ready to use for evaluation of projects. Most of the core projects were evaluated with the criteria. At the same time, three thematic working groups were established within the LOCREGIS project. The idea of working groups was to define and complement on specific areas the content of the best practice criteria (partnership, dissemination, etc.) and to help in their further development. Application of the criteria to different kinds of projects was another subject discussed by working groups.
External experts have also participated in the developing of the criteria. Especially comments of experts working in regional planning have been essential to the work.
In the beginning of 1998, an electronic evaluation form was available on the web site of the LOCREGIS project. Since then, the open self-evaluation has been available in the LOCREGIS database.
The best practice criteria work continued in LOCREGIS II. Instead of making an inventory of the experiences of the projects and project leaders, the new project improved the partnership with regional development experts and combined these two approaches by using as a basis the preliminary versions of the work.
The main difference in the new version was the restructuring of the subcriteria to better meet the needs of the project selection and providing more useful practical examples for profiling. In addition, some parts of the criterion E (Regional development policy) are totally new.
LOCREGIS II working group 1 prepared two versions of the criteria, which were tested by the regional partners. The testing period caused some changes in the ways the criteria were presented because of some misunderstanding in the original formulation. This is a phenomenon that can never totally be avoided.
Instead of using points 1-5 evaluation, the new version is based on the Yes/No alternatives and therfore the graphics provided in Internet are not visually as good as in the old system. On the other hand, the new scale provides easier profiling and wider further possibilities to present the results.
This handbook has been prepared to shortly describe not only the best practice criteria but also to give some thoughts for their use in different situations. At the same time we must keep in mind that this kind of method contains plenty of restrictions:
The criteria should describe such elements that have a positive impact on regional development in local and regional information society projects. Moreover, not only in the area, where a project has been implemented but also in wider context. It is clear that no national or EU level financial instrument can solve all the problems or financial needs in every region or town, also repercussions directed to the other areas are needed in order to get positive results. For example, a project leader cannot see the project as a part of regional development in the field of information society. He/she may not realise the importance of transfer of the results or is not interested in wider implementation. This is one of the first requests from the public investors.
One of the most important aims of the LOCREGIS best practice criteria has been to develop the criteria for decision-makers of regional development programmes and projects belonging to them and that is why they can differ a great deal from the needs which are locally considered important. The main use is to evaluate completed projects. But it can also be used in guiding the projects, transferring the best practises, regional development and channelling funds to proper areas of actions.
The best practice criteria are not planned to describe which project is good methodologically or as regards to its implementation in general. Of course, these elements are also important in order to achieve the expected results.
During recent years, the development of benchmarking method based on soft values and planned for a target area has become increasingly important. Although this was also a case in the development of the LOCREGIS best practice criteria in the beginning, another point of view was found during the testing of the criteria. Despite certain restrictions, the criteria can be used more widely, e.g. in regional development, and can give a brand new aspect of information society projects.
LOCREGIS - best practice criteria are created to evaluation/profiling IS-projects and it serves the needs of regions and regional development e.g. by clarifying a connection between information society and regional development. Many different users and target groups can use it.
Why to use? Simply for better efficiency; both in planning and targeting the project. Criteria helps in benchmarking similar projects in different countries and regions and it can also open ways to potential partnerships.
Different areas benefit from the experiences of others – a good idea can be adopted by others and ideas not so good can be dumped before serious damage has occurred.
The best practice criteria have been developed keeping in mind many uses and users especially in the field of regional development and planning. Purposes and targets of the users may differ but when it comes to projects concerning information society and regional development the use of LOCREGIS- criteria and database is one of the first things to do.
Target groups for the criteria could be as follows:
5. Organisations who are responsible for the regional development and especially people who prepare the regional development programmes or participate otherwise in regional development, including financiers.
Projects co-financed by the European Regional Development Fund always aim at supporting regional development in different ways. However, there are major differences between the Member States and regions about how these objectives are understood in decision-making process. If factors that are significant for regional development will be considered already in the planning process of a project, together with the demands of Objective programmes and special features of region, a positive decision of co-financiers is more often received.
During the inventory stage of the LOCREGIS-project, many project applications of the Objective programmes were examined. It became clear that many projects were planned to have short-term repercussions whereas the original idea with only minor changes would have given opportunities for better regional results.
Not only can the criteria be used for preparing wider strategies with no direct application for financing, such as information society strategies for municipality or sub-regions, but it can also be used in drafting project applications.
LOCREGIS best practice criteria have been specially modified to use both ex ante and post evaluation, as well as on-going evaluation during the project implementation.
In addition to the planning stage, the criteria can be applied to support on-going evaluation of a project and monitor the expected results. There were some cases during the development of the criteria when a test evaluation was made in planning or in the beginning of the implementation stage and then later, almost at the end of the project. Differences between these two stages varied a lot at least in emphasizing different aspects of the project. A reason for this could have been the impact of the learning process during a project in addition to subjectivity of the method, but also the fact that the goals of local and regional information society projects often change with or without any plan.
The criteria help in acquiring such information that helps in analysing these needs for change and planning of future project actions.
Profiling the projects gives advantage to regional authorities to view their influence in regional development. They can guide regional development for example by starting new types of projects or serving ongoing projects by feedback information or extra resources. It can also give a view from the regions position in the current IS and give positive development impulse to foster IS development. By using inventory of IS projects such as LOCREGIS database and evaluation forms (available on Internet) profiling is easy.
The following uses can be found for project evaluators and decision-makers who will have support information for funding decisions:
For Priority definition
For example because of Objective programmes, a pre-decision can be made already before the period of application, on requisites of projects to be recommended. This can be made by specifying descriptions of the LOCREGIS criteria and in addition by giving different projects (e.g. networking of SME’s, telework, public services) different descriptions of requisites.
For Comparison
A comparison of projects to be evaluated can be made with the best practice criteria especially on the basis of priority definition made in advance. Naturally, the emphasis of comparison can only be made on possible impacts on regional development.
The use of LOCREGIS best practice criteria can give valuable information to select the right applications. Anyway, they are not recommended to be used as the only selection criteria.
Normally background documents (applications, reports) do not provide enough information to enable the perfect use of criteria but some of the subcriteria are difficult to verify if that information is missing.
The recommended methodology is to discuss with project leader after becoming acquainted with the project documents and to go through all the subcriteria. This discussion should produce the final information if any certain subcriterion is or isn't inside the project.
In some cases, the discussion needs a little preparation. E.g., it should be clear what are the regional development goals before discussing the compatibility of the project with that policy. This is a feature that is not always clearly recognised in the regional development organisations.
Even the project comparisons and improvements in the project ideas are important goals of LOCREGIS best practice criteria, along with the use of the criteria as a part of the LOCREGIS toolset.
The different areas and projects are available for profiling through LOCREGIS database, which makes it possible to compare projects and find information about these different projects and regions. The inventory of projects starts by filling in and sending the evaluation form and/or adding new project to database. The database needs certain basic information about the project such as : state, region, EU-programme and sector. In the LOCREGIS database the project must be implemented on a certain region/regions by a local and/or regional activators. It can also be a national or international initiative. Evaluation form and other database information are available in LOCREGIS homepage http://www.locregis.net .
When the information of a project is sent to LOCREGIS database, the project is ready for profiling. The evaluation form uses the questions based on the best practice criteria and gives tested projects a profile, which can be compared with other projects. Using the information from LOCREGIS database, it is possible to produce regional information society profiles with combinations of different backgrounds.
The criteria do not describe special local features either. Instead of that, the LOCREGIS best practice criteria find their place in completing general guidelines and comparison in the way described above.
As mentioned before, the LOCREGIS best practice criteria are meant
for profiling of local and regional information society projects and, above
all, for evaluation of their impact on the regional development. If they are
used as made available, without changes, an advantage is gained: a project can
be compared with other projects. In
addition, a good overall view of different projects will be obtained by using
only one type of criteria.
A discussion between the evaluator and the project leader/worker has turned out to be the best way to evaluate projects. However, the evaluator should well know the subject in order to be able to make his evaluation as objective as possible. This concerns especially the criteria with relative values such as innovation.
Even though the criteria can be used for ex ante and post-evaluations, the results can also vary a great deal depending on the implementation stage of the project. When a project is almost at the end, there is always more information available than when it has just started and especially in the large umbrella project, the implementation environment can be very different than it was in the planning phase.
In the same way, evaluation results might change depending on the evaluator’s point of view. If the evaluator has an inside view of the project, evaluation results are different than in the contrary case.
The following five main criteria and their subcriteria can be used as indicators and meters in the evaluation system. All main criteria are formed by LOCREGIS project. Detailed introduction of each main- and subcriterion contains examples of how project can/should influence the surroundings – to profile how successful the project is.
Giving a more practical view, each subcriterion is presented with an example of North Karelia evaluation form. Examples show in practise how each criterion is getting its matrix values, variables and indicators – and what the regional demands for successful implementation are.
This first criterion is about the impacts of the project on local and regional attractiveness and ways to increase it. This is the key point that many information society projects funded by the Structural Funds try to achieve.
Local attractiveness includes many other things than just the possibility to use Internet and other network services - the quality of life in general. Information society can offer solutions to areas, which now suffer from location disadvantages and are losing skilled and young people to the other areas, which offer better employment possibilities.
New business opportunities may create new enterprises, and new jobs attract skilled people to the area, also awakening existing creative potential. All these things together can improve regional image and give the region better regional self-esteem. If local projects can decrease regional problems and disparities, regional and local attractiveness may increase and give the area more hope of a better future.
An attractive region has an infrastructure based on the needs of the people and organisations. The existence of this infrastructure is a good marketing tool over regional borders and gives the area better image.
Project can be a tool for building new regional infrastructure. Not just things directly related to the telecommunication but which generally help to form new possibilities for local and regional people.
It can also help people to gain both technical and intellectual abilities to use telecommunication.
Projects can influence their surroundings for example by:

By offering more user-friendly services and infrastructure, a project can create awareness, new local and/or regional content along with activities. Technology should be up to date, easy to use and its further development should be based on the needs of people.
Locally interesting content in web pages can bring new users and increase demand for services. If local people find telecommunication a suitable forum to them, they will naturally keep using services and infrastructure. A good information society project creates services that are useful and easy to use and take into account the needs of end users.
Projects can influence by creating for example following things:

Information society has many positive effects on environment. It can offer better and/or new services, a more equal society and higher level of education. It can help in decreasing specific regional problems/disparities, and, besides Information Technology is pro-environmental.
Regionally connections through telecommunication might open new ways for employment by making certain services available. E-mail services, for example, can help people who do not physically live near the centres and services and offer elderly people more independent years in own home. Opportunities for self-education are also better and drifting away from society is possibly prevented.
A good information society project supports for example these following targets related to a higher quality of life:

New business opportunities might turn up during the project. A good project can create or attract new enterprises in the region, reducing location disadvantages and improving employment. A project might raise interest among private enterprises and involve them in co-operating with all kinds of side effects.
Good project can promote:
All these things, individually and together, can create regional economic improvements by offering new jobs and e.g. turning a former agricultural area to an area with strong information society involvement and technical advancement, which no longer suffers from its unfavourable location.

When a project has been properly prepared and marketed, the achievements of the project may attract skilled people outside the region or help trained people to stay in the area.
People must have the skills and tools with them so they can work and express themselves. With better skilled people, the region can be attractive for enterprises seeking new opportunities.
A well-organised and prepared project must take into account the possibility of increased regional attractiveness and support regional attractiveness by its own work and support these examples:
One of the most urgent things on many regions is to keep young and trained people in the area by offering jobs, education and other possibilities through information technology.

The project must exploit national (or international) media to improve its visibility/regional image.
Projects can influence for example by:
.
The LOCREGIS subcriteria can be implemented by evaluating how many times it has been featured in local, national or/and international media. Higher visibility can be part of the increase local esteem just by improving the image.
All these efforts can also improve regional self-esteem.
Analyses of the local and regional information society projects produced the experience that innovativeness can be a crucial factor in the regional development effects of the project. Innovation means that the project has something new to offer or some new ideas about existing structures of society or technology.
A project can be characterised by this LOCREGIS sub-criterion concerning the levels, content, tools and specified target groups, i.e. the type of innovativeness it creates.
An innovative project stands out in critical mass of projects and supports the intellectual life of the region. A project can also be innovative in reengineering the existing structures; in the public government, networking the SME’s, building up virtual enterprises, creating new business culture, by being socially innovative and including innovativeness in implementation and applying new forms of leadership.
A project can be innovative in many different ways and levels e.g. inside the own region/geographical target area

A project can provide innovativeness also by crossing regional borders. It can even be innovative on European or global level. A project can create different and transferable tools (concepts) acceptable on different levels, e.g. regional, national and international level. Innovativeness is difficult to measure and it is useful to use expert evolutions related to the project’s innovativeness.
The objectives of the project can be innovative only concerning its target group.
A good project can for example:

An innovative project can create new tools and concepts for the existing structures for example:

Innovativeness may also be a new kind of implementation. E.g. the leadership (or management) of the new project is innovative or it is using new kind of co-operation tools in the partnership.
The project can also be more like a learning process.

Level of challenges can also be one part of the innovativeness. That is the implementation includes some kind of positive development risks, which must be solved during the project. In many cases, too safety world can decrease the problem solving approach.
The same subcriterion can also be described by saying that the project has potential for exploitation

The right partnership can in many cases improve the regional development effects of the project. This does not only mean that the right actors are participating in the project. The use of the experiences gathered in the other regions is also important.
A good and diversified partnership increases a project’s repercussions and exchange of experiences and strengthens common goals of the regions. Project partnership takes into consideration all natural and/or important partners both in local and regional sectors. Possible international partnerships must be taken under consideration and support them if they are appropriate.
If co-operation is successful, the project can create partnerships in a long-term and independent business relationships between the local and regional partners. Private and public sector can also get involved in the project and find appropriate partners among themselves.
Successful project partnerships can be for example:

Because economically region develops mainly only through the private sector, the involving of the private sector in the information society projects is important. A public-private partnership is not a must in every case, but also without a clear task the co-operation of the both sectors can be fruitful.

The creation of new networking platforms in the project can support the co-operation in many other things. This can be done by supporting development of the structural partnership.

A good information society project takes the competencies of partners in account – optimises them for example in:

Transregional partnership means also transfer of experiences (and expertise) and can provide easier way to achieve the results. This transfer can also be implemented by consultative guidance.
In some cases, an administrative transregional project partnership in the beginning has developed to a co-operative network of the local enterprises, which can also be taken into account in the project planning.

The existing partnership provides more effective way to co-operate in the future.

For the regional development, the strategic planning is the main part of the project planning. Without proper charting of necessities and needs, regional development targets and business plans, the project can only be implemented inside its own world.
Strategic planning has to take into consideration the different partners in marketing the project to the targeting groups. Marketing should be planned and accomplished keeping in mind the different partners and not to forget the involvement of the end-users. One of the useful tools, which can help controlling the project, is an ongoing controlling system. Partners hold a special position when projects have to examine the possible benefits and disadvantages the different partners might have.
The subcriteria of strategic planning are formulated for the use in ex ante evaluation, which mainly makes an inventory of certain existing documents. In post-evaluation this can be replaced by the evaluation of the real activities (e.g. in the dissemination of the results).
Market analyse prepared before the project starts can include following elements:

A good project should plan the way the end-users participate in the project and how their needs are noticed. In many projects, this can also be part of the dissemination.

Both benefits and possible disadvantages have been examined by the project. Well motivated partners do better co-operation and when there is knowledge about possible risks, possible set backs should not ruin the whole project or relations between partners. Win-to-win principle can often be a good methodology in the commitment of the partners

Most of all information society projects must be flexible and take into consideration the fast changes in the surrounding environment. Even the implementation period would be only one year, the time from the first ideas to the final report can, with all the needed bureaucratic handling, be three times longer.
Therefore, the projects being effective in the regional development must have some kind of plan-do-control-act – tools to be sure that the unscheduled changes will be adapted.
When the controlling system is available and in proper use, the project leader and sponsors can evaluate the on-going project and know more about the used and available resources. Best value from controlling system is gained when the system is used with objectivity and perhaps by independent evaluator.

One of the strategic planning tools is the survival plan, which will list the possibilities and ways to produce permanent results. In many cases, this means a business plan after the public funding has come to an end.

Dissemination of the results is one of the most crucial duties of the local and regional information society projects funded by the Structural Funds. Dissemination is often mixed with the marketing of the project, which of course can also be one part of the dissemination.
A dissemination plan must include all the activities needed to make the results available and usable.

Regional development policy criterion includes the features that are important in the decision-making process of public funding, but which have less importance in the project team.
Very rarely, the regions decide to provide all public funding to one big information society project and because of several funding sources and decision-makers this is also not possible in most cases.
That is why it is important that the same common objectives are supported by different measures and projects. LOCREGIS database includes examples, where also R&D Framework Programme projects have a role in the implementation of the regional information society policy.

Connections to the existing clusters provide synergy effects. The clusters can be regional (regional information society initiative or Chamber of Commerce) but also national (Schools in Internet) or even European (Telecities).
One important link can also be with the regional competence centres.

The transferability of the projects can mean that :
Ø It is producing transferable know-how
Ø It is producing results that help others to continue
Ø Because of its results, the implementation of future projects is cheaper
Transferability can also cover tools, terms and methodology produced by a project. One feature of the transferability can also be that it is possible to fund using different sources (e.g. implementing an originally Objective 1 project on the region without being able to use Structural Funds)

A spearhead or key project opens the way for new projects or it can even be a prerequisite to other projects.
A spearhead project can also create the needed infrastructure, awareness or critical mass.
A clear spearhead project is rather a process than a project, and its feature can also be remarkable spin-off effects.

Regional coverage is a feature that varies a lot by separate projects. Some regions implement one or two big umbrella projects, but some others use the same resources for several village projects. The basic idea of this subcriterion is not to tell that only geographically covering projects should be funded. It is more meant to highlight those two political extremities in the LOCREGIS regional information society profiles.
Instead of regional coverage, also the sectoral coverage can be used.

For example, leverage effect can be recognised by following features:
All kind of other
financial snow-ball effects
One of the main products of the LOCREGIS II project was the regional information society profile. This innovative product of the project combines :
· Regional background information
· Regional information available in the Internet
· The project database, and
· The project profiling information
· Analysis of the region and its potential.
At the end of the LOCREGIS II project, the regional profiles cover
· 30 countries (besides Europe also Canada and the United States)
· Over 400 regions.
The goal of developing the information society regional profiles has been to create a tool that can provide a picture of a region, that is as reliable as possible, with regard to its information society development in the European information society landscape.
The potential uses or applications of the LOCREGIS regional information society profile are varied. In general, the profile can help placing a region and its information society development on the map of all other European regions. However, it is also possible to use the profile as a tool for monitoring EU programming periods and their results in regions. The information society profile can be used as a tool in assessing the development of a region against itself, or other similar regions. The profile can also be used to provide a view of the extent to which given sectors in the region have been covered as far as information society development and projects go and also the extent to which the implementation of the ideas and projects has been innovative. The profile also provides information on what actors have been active, and which have been less active in the region’s information society projects. Finally, the LOCREGIS database can be used as a marketing tool for the region or for finding new project partners and learning from the ways other regions have solved particular problems.
The potential users of the regional information society profiles are
· Regional decision-makers and developers
· Local and regional authorities
· Project managers
· Financiers, and
· Enterprises seeking new locations.
The regional information society profile has been described as an especially useful tool and source of inspiration or ideas and examples by regional developers in the CEE countries who are planning regional information society strategies and developing information society projects in their regions.
The LOCREGIS information society regional profile is formed from the following elements:
· Regional background information
· Regional Information in Internet (like tourism portals)
· Project database
· Project profiling information using LOCREGIS best practice criteria
· Analysis of the region and its potential
The results of the project profiling (evaluation) are presented in relative terms. The profile is providing a regional list of all profiled/evaluated projects, where on the top is the subcriterion, which has most often been evaluated with YES on this region. E.g. if every project has made a dissemination plan, the value of the subcriterion D.6 is 100% and it is first on the list.
On the contrary, the subcriterion with the most NO – answers is the last one.
Because covering evaluation of the projects is an impossible task, this list is providing only complementary information in the regional profiles and helps in some cases to recognise some regional weaknesses (most often in transregional partnerships, innovativeness and geographical coverage).
LOCREGIS database provides the tools to profile the projects. However, this feature is not in the open use because it has been agreed that the management should be in the hands of the regional government.
In the open part of the LOCREGIS tools in Internet, any project can be evaluated and even the toolset includes an open database for project ideas and proposals, which allows anybody to test the use of the tools.
In LOCREGIS I the scale from 1 to 5 was used to evaluate different subcriteria. This methodology was providing results that were easy to compare and to visualise[1]. On the other hand the experience was that this kind of scale was difficult to use and also increasing the danger of giving invalid results.
In LOCREGIS I the main aim of the evaluation was to find best practice examples. Because this was not anymore the main objective in LOCREGIS II, it was agreed that instead of an evaluation, some sort of profiling of the projects was a better and easier method to provide information for the regional information society profiles.
This last profiling methodology is also used in the database tools. The user must decide if the project has a certain feature described in the subcriterion (YES) or if it does not (NO).
The experiences of the participating regional decision-makers were positive. This simpler methodology was found to be at least as valid as the previous one.
The picture on the next page is describing the project profiling in the LOCREGIS database. The database is giving a list of the projects by requested search criteria. Choosing the appropriate project will give a profiling form, which can be filled with YES/NO – profiling. The last phase is a picture with profiling results.
The open Internet part is also providing the possibilities to search the profiling results by
The limited number of profiled projects outside the target countries and main funding sources can produce results that are not at all valid.
The main aim of the RISI regions has been to formulate a regional information society strategy with implementation plans. RISI has been funded by the innovative actions of the European Regional Development Fund and European Social Fund.
NOKIS was the first RISI project of the Finnish
North Karelia region. This work continues in the years 1999-2001 with NOKIS II.
During NOKIS, main goals in North Karelia were related in building infrastructure for information society and other general RISI goals. NOKIS II continues with the same topics but also with development of the regional development tools.
North Karelia worked already before the LOCREGIS partnership with the regional information society indicators and achieved very innovative results. The participation in LOCREGIS provided the possibility to co-operate with other regions and produce results that might at least in some degree be comparable.
All criteria have their own area in the evaluating process. Many of the things, which are important in the process, must be found out by an independent evaluator because people who are involved in the project cannot evaluate the project with objectivity, no matter how honest they might be.
Evaluation system is basically simple and easy to use but when the user does not know the basic procedures and principles, it might seem too sophisticated and not suitable for field use. Everyone can fill up the forms and collect the basic information, which is collectible without the expert panel. Basic work is simple but in the next step, which requires knowledge of matrix variable values and how to measure these values, the evaluator must know and understand the basics about the criteria and more than basics about the evaluation system.
NOKIS II has been assigned as one of its tasks the development of an evaluation system for information society projects. This system will help in discerning the successful projects and will forward their best practises to new projects and permanent operating structures of information society.
The system aims at adequate objectivity, simplicity, economy and realism. This means that the majority of the information used is normal follow-up information produced in the course of the project's normal operations, evaluations being done by an independent expert panel or information otherwise easily accessible. The system evaluates completed or ongoing so-called IS projects. The common features are the use, adaptation and development of information technology.
The structure of the evaluation system is based on version 4 of the LOCREGIS criteria for the profiling of information society projects. This criterion is divided into five main criteria and each of them, in turn, into six sub-criteria. The actual evaluation of the projects is based on the sub-criteria, and the main criteria can be used in local reporting, e.g. in the analysis section of LOCREGIS projects and their information society profiles.
The LOCREGIS criteria are mainly designed for general evaluation on regional level in Europe. That is why the evaluation system for North Karelia IS-projects adapts them for evaluation using a single project as its unit. The general criteria and indicators are adapted to local meters and within these meters, in turn, locally exploitable project estimates are analysed. On local level greater accuracy in evaluation is achieved since the number of projects is smaller and they are physically closer.
During the process of developing the NOKIS evaluation system, a scale of 1-6 was adopted mainly because every main criterion has six sub-criteria. Matrix values for project meters and profiles were developed, keeping also in mind that every criterion has a maximum matrix value of 6. For example, “attractiveness “ has 6 sub-criteria and matrix value 1.0 each.
By this kind of matrix values, it is easy to measure a project’s success by the indicators based on LOCREGIS best practice criteria. The required information for profiles is gathered by discussion between evaluator and project leader/worker. The evaluator has questions on a form, which is filled during the discussion. Results from the filled-in forms are transferred to profiles by using normal calculation tools in PC.
The implementation of the evaluation system is co-ordinated by the NOKIS II (North Karelia towards Information Society) project of the Regional Council of North Karelia. The system can be exploited in e.g. the steering and developing of the projects as well as channelling funding. The main use for this system is to evaluate the completed projects and use the results for Best Practice transfer. The system can be used by regional development officials, by those who are funding the projects, by the organisations implementing the projects and last but not least by the project organisations themselves.
The basic information required for the system is collected by:
1. Secretarial work
The evaluation system has a secretary that makes sure that the system is functional and practical. The secretary collects the necessary basic information from the project documents, funding reports and by interviewing the actors. The secretary takes care of the guiding and operation of the expert panel by providing them with the material required and later by collecting the material from them for summarising.
2. Project itself
Projects routinely produce different follow-up and action reports where several indicators describing the project activities can be derived from. When necessary, projects can be asked to give more detailed information through either interviews or questionnaires. The so-called sponsor ´s right for asking for necessary information can also be used. Useful information may be available from other evaluators and evaluation systems as well.
3. Evaluation by an expert panel
Many of the indicators in project evaluation are – at least when measured with academic objectivity criteria – difficult, laborious and costly to measure. That is why this system settles with using an independent expert panel in judging such indicators that are hard to measure. For example, the LOCREGIS criteria B. Innovativeness is impossible to measure explicitly and/or mathematically. That is why an expert panel using a scale measuring difference in quality must do the evaluation. As a rule, the expert panel meets through e- mail and through material delivered by mail. The project materials are sent by mail and e-mail and the evaluations are sent back to the secretary by e- mail.
In North Karelia two information society projects have been
evaluated by using the evaluation, system based on the LOCREGIS best practise
criteria. These projects are: Learning Upper Karelia (Oppiva Ylä-Karjala)
and Telework in Lieksa workers and market (Lieksan etätyön uudet markkinat
ja tekijät).
Learning Upper Karelia has been one of the local spearhead projects in North Karelia. During its implementation 1998-2000, the project provided local access by application and training. It has also been transferred later to other regions in Finland.
Learning Upper Karelia can be seen as a forerunner in information society and it has greatly increased the use of telecommunication systems among local people.
The other evaluated project is promoting telework.
The first diagram shows Telework in Lieksa workers and market project profiled by evaluation systems LOCREGIS best practice criteria based meters and indicators. The same values of Lieksa are presented in tabular form in the picture.

![]()
