Research and development project assessment and social impact

Abstract

Nowadays, organisations increasingly need to adapt to the fast evolution of markets and societies in our globalised world in order to be competitive. Therefore, it is essential to take the right decisions when it comes to invest in research and development (R&D) projects. However, an issue that has not been given much attention is how to measure the social impact (or return) of R&D projects. In this exploratory study, the findings of an analysis of how R&D projects are assessed and selected, including this social perspective, are presented. The methodology which has been used in this research includes both interviews and analysis of the data obtained through them. The major finding is that in the current situation the social impact is not taken into account, but is growing the awareness of this perspective among different types of organizations dealing with R&D activities.

Keywords
R&D projects; Social impact; Survey analysis

1 Introduction

Investment in R&D projects is an essential element if we are to increase competitiveness, especially in the case of technology-based companies (Bitman & Sharif, 2008). But when funds are limited and there are several alternative projects, it is necessary to define a methodology that enables us to balance the different aspects that must be taken into consideration in the decision-making process. In fact, the choice of investment projects is an important strategic decision for all businesses and, as all R&D activities begin with an idea, the most important decisions companies have to make are those related to financing and developing new ideas and projects as well as those linked to determining whether existing projects should be set aside or continued (Henig & Katz, 1996). Several studies have revealed that the use of traditional financial techniques for project-assessment purposes is not the most suitable one when it comes to analysing investment in R&D (Chan et al., 2001). The use of these techniques consists, essentially, on estimating the investment's cash flows and applying evaluation methods to assess their viability (e.g. net present value, NPV, and internal rate of return, IRR). This procedure implies that the costs and benefits associated with investment are easily and objectively quantified. However, this cannot always be achieved, particularly in the case of R&D projects, where three different types of benefits can be distinguished

(Adler, 2000): strategic, measurable and intangible. For instance, if we take the example of intangible benefits, it should be noted that they are difficult to quantify but may nevertheless have a significant impact on return on investment.

As emphasized by Henig & Katz (1996), the NPV of a project in basic research is virtually impossible to calculate, and the same goes for R&D projects given, namely, the uncertainty and risk involved in those investments. In fact, innovations based on new technology, or addressing new user needs or creating new markets, are difficult to evaluate, especially in the early stages of product development, due to many uncertainties and risks (Vandaele & Decouttere, 2013). Moreover, R&D activities usually imply a long lead time with uncertain results in terms of technical implication, risk, life time expenditure, resource usage, and market outcome (Hassanzadeh et al., 2012). Companies are looking for new technologies, yet to be developed, where it is impossible to extrapolate probabilities from past experiences. Companies must make all their assessments while minimizing any information leaks to their competitors. On the other hand, there is a noticeable need for companies to include in their decision-making process specific criteria of non-financial nature (e.g. strategy, flexibility, quality, social returns). In fact, these non-financial aspects are particularly important in the new industrial environment in which firms operate, where new technological developments tend to occur more rapidly than the evolution of project-evaluation techniques (Brownell & Merchant, 1990). Therefore, R&D managers need new R&D decision support tools in order to take into account in a proper way the inherent complexities of R&D projects, and provide the decision maker with a more realistic representation of a R&D project in its surrounding environment (Hassanzadeh et al., 2012).

In recent decades, increasing competition, reduced life cycles and globalisation of markets have highlighted the interest of measuring the importance of research and development (R&D). This issue has indeed become a major concern for R&D managers (Chiesa et al., 2009). However, the different contributions from the study of the technological and evolutionary aspects of the economy question the need to incorporate social criteria in the field of scientific research and engineering. These contributions allow us to establish a new concept of technological change based on the co-evolution of technology and society (Rip & Kemp, 1998).

Therefore, in this paper the results of an empirical study focused on how R&D investment projects are evaluated in the wood and furniture sector of Galician region (Spain) are presented, emphasising the social return perspective. The high capacity of the forests of Galicia and its economic potential makes the wood sector as one of the pillars of the Galician economy, lagging behind the automobile and textile industries/ sectors. The wood and furniture sector in Galicia has an annual turnover of 2,259 million euros (Barreiro, 2012). It also represents 3.7% of the Galicia GDP, employing directly 26,000 people and accounts for 25% of total R&D spending of this region. However, there is an imbalance in the value chain since Galicia generates almost half of the wood produced in Spain but has a much lower weight in the processing of the wood accounting only for 10% of the total. From data of National Institute of Statistics (INE) one can see that the percentage of companies from the wood and furniture sector performing R&D activities is 10.24%, and the percentage of personnel dedicated to R&D is slightly less than 1%. The total amount of R&D expenditures by those firms was around 39 million euros. To foster R&D activities and to ensure the future of the Galician wood and furniture sector the Wood Technology Centre (CIS-Madera) and the Wood Cluster (CMA) were created, in order to promote an increasing cooperation between business companies, technology centres and universities. This will allow greater innovation, new product development and differentiation, and the search for new markets.

The methodology which has been used in this research includes both a literature review and interviews, analysing the data produced. R&D managers of three companies and also of technology centres, all related to the above-mentioned sector, academics from the University of Vigo and trade union members, as well as an agent of public programs of R&D, were interviewed, with the aim of providing a wider view of the selected subject. As such, attention has been given to different points of view on the chosen topic.

The remainder of the paper is organised as follows. Section 2 gives a brief literature overview about R&D projects evaluation and social impact. Section 3 describes the empirical methodology followed in the study. Sections 4 presents the results achieved. Finally, section 5 draws the main conclusions of the paper.

2. Literature review

In this section, a number of studies on the importance of investing in R&D projects and the selection and evaluation process are reviewed, emphasising the role of non-financial criteria in the evaluation process.

Investing in R&D is of paramount importance for increasing a company's competitiveness, especially those in technology industries (Bitman & Sharif, 2008). Given the increasing unpredictable business environment in which firms operate, R&D managers are confronted with new challenges because market uncertainties found in the development of new products and services are seldom systematic and may change over time (Hassanzadeh et al., 2012).

R&D portfolio decisions are, mainly, made based on financial indicators, such as NPV or IRR (Ghasemi et al., 2011; Nigro et al., 2014; Vandaele & Decouttere, 2013). However, several studies have shown that the use of traditional financial techniques in project evaluation was not the most suitable for the analysis of investment in R&D (Chan et al., 2001). In the case of R&D projects, the risk of additional factors (e.g. technical uncertainty) to the natural uncertainties of the market induces companies to underinvest in R&D when assessments are made using the conventional financial techniques (Leite et al., 2012). In fact, traditional models for projects investment appraisal may rise some difficulties when there is uncertainty in the cash flows, forecasts or volatility of key variables. This has led many authors to question the adequacy of those techniques when uncertainty is the key factor in determining the viability of a project (Barroso & Iniesta, 2013). Furthermore, the selection process of R&D projects faces various difficulties, namely how to measure the impact of the R & D projects, and which selection process optimisation to use among projects with multiple, and sometimes incomparable, performance indicators (Duch-Brown et al., 2012).

Therefore, those investments should be assessed not only adopting a financial perspective but also a qualitative perspective with an appropriate model. In fact, it is necessary to include various perspectives (e.g. strategic, analytical and financial) in the analysis and to find a suitable methodology which takes into account a range of different criteria that are to be considered when selecting which projects should be developed (Henig & Katz, 1996). One possible solution would be the adoption of multi-criteria methods which take into account a range of quantitative and qualitative factors when assessing projects (Zopounidis & Doumpos, 2002). In fact, Vandaele & Decouttere (2013) concluded for the real need of a R&D assessment decision support tool integrating several dimensions of analysis and to accomplish that need it is necessary to resource to multi-criteria tools to assess the impact of R&D projects on the company including several performance factors (Chang & Tzeng, 2010).

In this context, an important aspect that should be included in the evaluation models is the social impact (or return) of R&D projects. For example, Chiesa et al. (2009) examine the problematic task of assessing R&D results including the social focus. In particular, these authors explore the iteration between measurement objectives, dimensions of performance and contextual factors in the design of a performance measurement system for R&D.

The different contributions from the study of technology are concerned with the need to incorporate social criteria in the field of scientific research and engineering. These contributions allow us to establish a new concept of technological change based on the co-evolution of technology and society (Rip & Kemp, 1998).

The concept of social responsibility arises in the twentieth century. Although the term comes from the 1950s-60s in the U.S., it failed to develop in Europe until the 1990s, when the European Commission used this concept to involve employers in an employment strategy that would generate greater social cohesion. In the European society there were increasing problems related to long-term unemployment and the resultant social exclusion.

Over time, social responsibility is gaining importance and companies are trying to find a proper balance between economic profitability and social responsibility.

In order to find out what the major policies, strategies and practices that are being developed regarding corporate social responsibility (CSR) and how to evaluate them, Galician companies created the Permanent Observatory for CSR in 2010.

The growing interest in CSR has led to the establishment of new awards and recognitions, the proposed measurement indicators, the study of the perceptions of different stakeholders (managers, shareholders, employees, customers, etc.) or the inclusion of social and environmental criteria in awarding public contracts, which means that CSR is not a fad, but a new way of understanding the role of business in our society while obtaining financial, social and environmental benefits and improving the competitiveness of the company. These criteria include social impact on R&D management. This impact begins to be seen as a potential source of profit, as it increases the degree of consumer confidence and reduces the likelihood of conflicts among the different groups affected (Carroll & Buchholtz, 2009). Also in some R&D programmes designed to fund research one can find references to social aspects, such as the European Commission's Framework Programme nº 7.

From a standpoint that gives priority to social criteria rather than to economic effects, it is necessary to carry out a sector analysis and to identify measures and actions which may enable us to determine whether social return on investment in R&D exists. According to Ares et al. (2008), such factors can be considered to be either positive or negative effects of public investment in R&D.

Accordingly, the concept of social impact can be broadly defined as a combination of multiple environmental, socioeconomic and scientific factors which are often left out of traditional mechanisms for evaluating R&D (Moñux et al., 2006). Governments are increasingly trying to take into account social progress. For instance, integration of women and disabled people in the workplace is becoming an important issue in our days. This should be taken into consideration by companies when assessing R&D projects. It is necessary to analyse, to discuss and to synthesise to solve social problems, just in the same way as medical examinations are performed to measure health indicators and to make sure that our organs and senses work properly.

In conclusion, it is essential to invest in R&D and to make a proper assessment of projects based on multiple criteria, due to the competitive environment in which companies coexist. Therefore, the inclusion of social criteria for evaluation of R&D investment projects is highlighted in this study.

3. Research methodology

Before investing in R&D projects, companies should decide how to finance them. Funds can be both private and public. In some countries, public funding of R&D projects assumes an important role. Therefore, we have focused on the Galician Plan R&D&i INCITE (Xunta De Galicia, 2007). The main objective of this plan is to develop the research and innovative Galician potential in order to achieve positive results in social welfare and economy. To this end, the Plan should look up to the future and be based on social trends to anticipate any potential social changes that might occur. Figure 1 illustrates that the Galician innovative system consists of four agents.

We argue that methodological complementarity, by means of an appropriate adjustment of the different existing approaches in order to achieve a better research in which the objectives of the research themselves are fully taken into account, are essential when deciding which method to use.

In this study, both quantitative and qualitative research techniques were used as pointed out in Figure 2. For the former, a large number of questions are set in such a way that the opinions of those interviewed are those which are the first to be considered and then lead to further reflection (Olaz, 2007, 2008). For the latter, interviews were conducted, which is an essential element when selecting those common criteria which are considered relevant by respondents. That is why the methodology we have adopted is of mixed nature and is focused on the case study (Figure 2).

Our methodology includes the selection of a few interviews which consist of both open-ended questions and closed questions. In order to carry out this research project, we chose to conduct two-part interviews: on the first hand, we can find essential questions whose aim is to deepen on the subject and which are of great interest to the interviewer, who can add more questions during the interview if appropriate. On the other hand, we have used a standard questionnaire which enables us to compare the answers provided by different respondents and to quantify the obtained results. We have opted for this kind of interviews because we find it is the most flexible mean in terms of data gathering providing a highest response rate. It is, however, obviously a laborious methodology which demands much time.

Figure 1
Galician R&D system.

Figure 2
Methodology.

In most cases, we have opted for individual interviews. Interviewees were always representatives of the companies or institutions most involved with R&D funding. In one case there was a group interview in which several people of the same organization were included.

The first questions were closed, as we had designed them to provide the interview with a context. Nevertheless, some room for open questions was always given to interviewees so that they could express their opinions freely. This enabled us to compile extra information and views which were extremely useful for our research. After completing the interviews, there was a stage of information processing and speech coding. Recorded conversations were transcribed to paper, a laborious but very interesting task which would permit us to analyse the information clearly and to focus on interpreting data.

Regarding the structure of the interview conducted, it consisted of a three-part structure. The heading explains that the purpose of the interview is to conduct a study on the evaluation of R&D in Galicia, even though we had previously explained this to the interviewees in previous encounters. It states the duration of the interview (30 minutes) and also that the information provided will be only used for research purposes. The first part of the interview consists in collecting general data of the company in question, such as: the name of the company or institution; market sector and field of activity; contact person and his/her position in the company; year in which the company was founded; company's turnover; number of employees; position in the value chain (extractive, manufacturer, wholesaler, retailer, services); and other attributes (e.g. exporter, high performance, certified quality, audited, other).

The second part is directly related to R&D projects evaluation. Questions 1-9 deal with R&D performed by the company in order to learn about its situation with regard to R&D: number of years they have been developing projects, collaborators, employees involved, budget, previous projects, objectives, indicators and source of funding. Questions 10-18 are more focused on public funding, R&D plans and projectmonitoring. These questions enable us to find out the views companies have concerning their current plans on project assessment, by emphasizing social issues.

Finally, the third section of the interview consists of an optional questionnaire aimed at quantitatively assessing the current indicators drawn from the latest calls for R&D proposals for grants by the Galician Government (Xunta de Galicia) and the Spanish Ministry. We decided to combine technological, social and economic standards because of their current rating and also in order to avoid monotony. However, the main aim is to find out the main social criteria which are taken into account by companies when it comes to R&D projects. In fact, in the sector programmes of the Xunta's INCITE Programme, social criteria are not given much importance since the current figure is lower than 5% at a regional level and only the presence of women is considered to be a social criterion. Annex 1 presents the list of the criteria/ indicators presented to the interviewees, and in order to select them we proceeded as follows:

• Read the latest announcements from Xunta de Galicia's programme SUMA (INCITE) - since they are the most attractive ones for many companies - so as to choose those criteria currently valued;

• Read the latest announcements from the National Plan in order to choose those criteria which are currently being evaluated;

• Review of the criteria used by RESIPIDI (Ares et al., 2008);

• Review of the criteria suggested by previous studies (Carvalho et al., 2010);

• Review of the bibliography discussed in the study.

In all cases we selected the criteria which, in our opinion, would be more suitable for our study. The criteria were divided in two main categories related to the project proposal and its impacts. As for the project proposal, two sub-categories were identified: (1) the scientific and technological interest of the project and (2) the implementation procedure and management capacity. As for the project impacts, three sub-categories were identified: (1) social and environmental; (2) economic and market and (3) scientific and knowledge transfer. Each one of these sub-categories was divided in a set of sub-criteria that can turn easier the evaluation of projects against the main criteria.

The questionnaire has been divided into two parts. In both cases a subjective numerical scale or Likert scale (1-5 points) has been used as appropriate to help us reach the two objectives pursued by this research. In the first section, we have tried to find out the assessment of current project evaluation made by those respondents who were familiar with or heavily involved in R&D funding and therefore knew the current criteria used for evaluation. In other words, our aim was to draw conclusions on whether R&D project assessment is being carried out in a satisfactory manner concerning the most recent calls which attracted the companies' interest. In the case of the interviewed companies, most of them resort to private funding so they decided to ignore this part of the interview due to their poor knowledge of the applicable criteria.

As for the second section, it has been designed to find out the score that the respondent would give to the different criteria to be applied when assessing R&D. This data enables us to draw conclusions regarding the criteria which are considered to be the most important parameters from all the points of view we have studied: business, research institutes, universities and unions. All respondents were in the position to complete this section of the interview as, although companies currently receive no public funding, they might wish to resort to it in the future. Therefore, all opinions were considered to be valid.

As we are dealing with R&D project assessment, we first visited AENOR (the Spanish Association for Standardization and Certification) website in order to look for certified companies in R&D Management, since they would be more likely to know the subject of research and could possibly collaborate with us. As a consequence, a company of the wood sector - and, more specifically, devoted to furniture manufacturing - was selected for our research. Moreover, we contacted the Wood Technology Centre (CIS-Madera) which is located in Ourense (Galicia). At first, the idea was to study the points of view of a company and a technology centre. After these interviews, we processed both the data and the conclusions drawn. As the information was too scarce to draw conclusions, we decided to try to contact other companies of the same sector as well as more technology centres. Moreover, we had the opportunity of interviewing a person who was linked to R&D funding at the University of Vigo, which was extremely interesting. Since trade unions are organizations formed by workers for the defence and promotion of their social, economic and professional interests, we found it would be suitable to contact the major trade unions in Galicia for a possible collaboration. Finally, it was possible to interview a program manager. Thus, this research enabled us to compare five different views on the subject: business, technology centres, university, unions and project managers themselves. Also, it allowed reaching a wider perspective of the subject than our initial proposal. In total, our collaborators were three companies of the furniture sector, two technology centres, OTRI (Research Results Transfer Office from University of Vigo), three trade unions and the program manager of the Xunta de Galicia for the INCITE programme (Innovation, Science and Technology programme).

4. Results

In this section the results of the interviews undertaken are presented, regarding each group of respondents.

4.1. Companies

The three companies perform R&D activities focusing primarily on innovation. Two of the companies are also certified under the standard UNE 166002 in management of R&D. This may be an indication of the forward-looking perspectives of these companies, given the increasing competitiveness in markets. The implementation of a system of R&D management would provide a number of advantages, such as: promoting R&D activities; provide guidelines for appropriate management; it is likely to generate proprietary technologies and patents; enhance R&D as a competitiveness factor; helping planning, organizing and controlling R&D units, which results in a saving of resources and improved motivation and involvement of employees.

It is found that the three companies are working with the Technology Centre (CIS-Madera) on innovation and treated wood, despite being, mostly, internal projects. They have also collaborated, occasionally, with universities and other companies.

The number of staff dedicated to R&D activities in one of the companies was less than 20%, and in the other less than 5% (one of the companies give no answer to this question). This numbers far exceed the percentage of staff dedicated to R&D in this sector which is down 0.5%. Also, R&D activities represent over 2% of companies' turnover, a percentage that exceeds the sector's average value (less than 1%).

In recent projects undertaken by companies, an environmental consciousness has emerged which is reflected in the preference to use recyclable materials and low pollution. This aspect can be linked to the environmental certification that one company owns. Another company reports the implementation of Lean Manufacturing processes, being the first company of the sector to implement it, for which public funding was obtained through the program to foster business innovation.

Regarding the major concerns (or goals) of the projects, they are, mainly, related to scientific and economic aspects. However, one of the companies reported, also, its awareness on the issue of training their employees to make them more versatile, which means that there is already a clear concern with the social impact, although not the principal one. Companies reported that projects' targets (or goals) tend to be met and the main purpose is the development of new products and processes.

When choosing a project, companies make an assessment of various ideas and evaluate the feasibility of the project taking into account the available resources. That is, a preliminary study is conducted according to several factors, such as: economic, market competition, financing and social responsibility. Moreover, one of the companies resorts to a weighting method with subsequent in-depth analysis of the project characteristics and resources requirement (e.g. time required, financial and material resources needed, environmental impacts, and analysis project risk). Regarding social criteria, companies stressed that investing in new projects ensures job stability, but this is not measured directly when developing the project.

As for how companies fund R&D projects, all of them indicate that they rely on equity funds but in the future could apply, also, to public funding for R&D. One reason why companies have not resorted to public funding may be due to internal issues or requirements, or bureaucracy, as was pointed out by one of the companies.

Regarding the follow-up of the undertaken projects, companies say that they monitor projects' indicators focused on meeting the deadline or timing of the project and the stage where the project is in order to take action to improve. Generally, they do not consider any social type indicator.

The companies were asked to give a score to the different criteria proposed to evaluate R&D projects. The results were analysed using the arithmetic mean as was done in previous studies (e.g. Carvalho et al., 2010). Table 1 sorts the most valued criteria for companies when evaluating R&D projects, based on the views/opinions of all three companies.

Therefore, we might conclude that among the most valued criteria are those that would have a social impact.

4.2. Technological centres

Given the nature of this type of organization, technology centres play an important role in R&D activities. It should be emphasised the strong degree of collaboration with other organizations including companies and universities. In fact, this collaboration is much more evident than the one between universities and companies.

Because of the condition of technological centres, the majority of their staff is dedicated to R&D activities. Directly, it involves about 70% of the staff, but adding indirect jobs corresponds to nearly the entire workforce. All R&D development is done by their staff.

Their main interests are economic or scientific projects, targeting higher profitability and improved technology to be more competitive in the market.

Table 1
Score attributed for the criteria by companies.

Unlike companies, technology centres have mixed funding: public and private. It proves once read the resolutions of several calls, the participation rate for public funds is much higher for technological centres than companies, except in some specific project, not publicly funded. With regard to programs technological centres are involved, they highlighted the Galician Plan for R&D, the Spanish National Plan for R&D and the EU Seventh Framework Programme.

Both centres have recognized success in their projects and believe it is due to their quality and experience. They also recognize the importance to track projects and make improvements, but do not take into account any social type indicator. We stress the fact that in one of the interviews despite the objective of the study was to analyse the influence of social type indicators, the technological centre continued to emphasize the importance of the economic objectives, being the social impact seen as indirect achievements of the economic ones.

Technology centres evaluate the Galician Plan for R&D as necessary and positive, and the need to maintain all objectives: scientific, economic and social. Regarding the assessment of the current evaluation criteria for R&D projects, one centre considers that it is rightly focused given that the projects' evaluation is done by experts. Another centre emphasizes the fact that sometimes there is a limit on how many applicants' projects can be awarded a grant, and they believe that scientific and technological criteria should be the basic criteria for the selection of these projects, prioritizing potential industrial application and experience of applicants.

Regarding the assessment that technological centres would give to the different proposed criteria for evaluation of R&D projects, it is verified that there is a tendency to value more the scientifictechnological and economic ones, than social criteria, such can be seen in Table 2. In fact, one of the two centres mainly highlighted two fundamental criteria that should be evaluated: the technological risk and the investment multiplier effect. Also, it emphasised the importance of not just applying R&D projects in order to obtain funding.

Table 2
Score attributed for the criteria by technological centres.

In conclusion, and unlike companies, in the ranking of the ten criteria most valued by the technology centres are, primarily, economic and scientific criteria.

4.3. University

As it is known, universities are centres specialized in R&D activities, and are characterized by the high degree of collaboration with other organizations, especially technology centres and companies. Given this nature, universities have a large number of research staff. Furthermore, universities have occasionally to hire more people or to award research grants to young researchers due to the fact that R&D projects applications have been approved.

Concerning projects' objectives, they spread by a large array of fields due to the universities' own nature. Therefore, objectives pertain to all areas, such as: scientific-technical, biological, social and humanistic.

Unlike companies and like technology centres, the University have a mixed source of funding: public and private. With regard to R&D programs funding it can apply, university highlighted the Galician Plan for R&D, the Spanish National Plan for R&D and the EU Seventh Framework Programme.

Regarding the monitoring of projects, it is said that in the publicly funded projects, monitoring reports are delivered, and in private funding true self-monitoring is done. Anyway, there is no social type indicator for monitoring. Just as was found for companies and technology centres, again becomes clear that some indicators are considered (as time or quality), but none of them addresses directly the issue of social impact of projects.

Despite making a very positive assessment of the Galician Plan for R&D, the University believes that there is always room for improvements and to value other aspects. Among them, is the importance of consolidate the research personnel in the system as a key issue to developing good R&D. This criterion, as we saw earlier, was also among the most valued by companies in the wood and furniture industry. Moreover, it is highlighted that, currently, there is no assessment to determine whether or not the initial objectives of the concluded projects were met.

In the interviewee's opinion more social criteria should be taken into account, namely as training issues or positioning in the global market. Drawing on his extensive experience (due to his responsibility in the University OTRI), we decided to perform a comparison between the way the criteria are currently valued and the score they should actually have. The results are shown in Table 3.

As seen in the table, of the 13 criteria that, in the opinion of the respondent, should change the score, 10 are of great social impact. Of these criteria, all should have a higher score except for university research (training) grants, which should have a lower score, since young researchers should have an employment contract. It should also be emphasised that the respondent suggested the inclusion of a new criterion based on the track record of the companies. In his opinion, it does not make sense that the same company apply always to the same program and not try to apply to other funding programs.

4.4. Unions

The script of the interview was mainly focused on companies, research institutes and universities. However, it was understood that the opinion of unions would be very interesting also, given their connections with aspects related to the workforce (e.g. training, employment), aspects of possible social impact. Therefore, the insights of three unions were collected. To this end, the script of the interview was adapted, and was focused more on unions' opinion about Galician Plan for R&D and the valuation criteria used.

One union was directly involved with the implementation of R&D activities (for example, in the area of renewable energy, and research that has to do with training, didactical materials, and simulators), so it was possible to conduct the interview under the initial script. It should be emphasized the collaboration with Universities' researchers and specialists to develop R&D projects. These projects were publicly funded. The interviewee underlined the lack of clarity of some of the criteria used in the public funding programs. In summary, for this union, the main features from these programs that should be positively evaluated are: to develop (and possess) own technology and not to rely on external developments; the contribution of research activities to the development of the industrial base and to create and maintain employment.

As for the opinion of another union, it focused on the new Galician R&D plan that was being drawn up for the period 2011-2015. According to the interviewee, some of the main ideas that should be taken into account in this plan were: awareness and training of staff (or employees); clarification of methods of technology transfer by the administration; encouragement of the production of more patents; change on public administration role becoming more active rather than passive (for example, by defining the priority sectors for R&D investment, and to enforce that objectives are met); inclusion of the vocational training issue in the R&D and favouring R&D research with applications.

Table 3
Score attributed for the criteria by Otri - university.

Table 4
Score attributed for the criteria by unions.

Criteria	Firm 1	Firm 2	Firm 3	Average value
36. Contribution to solving social problems (unemployment)	5	5	5	5.00
39. Consolidation / maintenance of existing employment	5	5	5	5.00
12. Adequacy of HR and materials to the work plan	5	5	4	4.67
27. Increased economic and job satisfaction of employees	5	5	4	4.67
32. Increased wage level	5	5	4	4.67
35. Contribution to solving social problems (quality of life for disabled people)	5	5	4	4.67
38. Corporate Social Responsibility	5	5	4	4.67
2. Scientific and technical level	5	4	4	4.33
42. Applicability and transferability of R?αμπ;D results	5	4	4	4.33
21. Contribution to improving the environment	5	4	4	4.33

Regarding the topic of the interview on the social impact of R&D activities, it was emphasized that the main objectives should be the creation of employment, the quality of jobs, employment security, and more training in R&D.

All three unions answered the quantitative part of the interview. Table 4 shows the ranking of the 10 most valued criteria. This ranking was based on the arithmetic mean of each union's score for the different criteria, as was done for companies.

Not surprisingly, the two most valued criteria relate to employment. This result was expected during the interviews, given that employment is one of the main interests of the unions.

4.5. Program Manager

Although the interviewed Program Manager is currently a regional government official, previously worked in R&D projects related to new materials, as well as on a technology centre which has collaborations with the University.

He believes that the main interests of R&D projects are scientific and technological, in order to create new products and processes, and suggests that the applicability of the projects is around 50%. In general, in the assessment of the projects, nor financial neither social indicators are specifically addressed, but it is requested a thorough and systematic preparation of applications for public funding to choose among the projects. In fact, to be successful, applications should be properly prepared, including, for example: the state of the art, objectives, applicability, methodology, work plan, and the budget consistent with the objectives and without inflation of figures.

The follow-up of the approved R&D projects is done through annual reports send by the companies supported and, also, through in situ visits. The program manager experience reveals that people usually work well in the execution of projects. One issue he believes to be very important is to know what projects' results are being applied in practice by companies and which wealth is being generated. These would be a very suitable indicator to know the effectiveness of the plans of R&D.

Although recognizing the Galician Plan for R&D as having an important role in revitalization and economic development, he argues that the more competitive projects should be more financially supported. Also, he criticizes the fact that the Galician Plan for R&D Table 5. Score attributed for the criteria by Program Manager.puts a great weight on scientific and technological objectives. In fact, to social and economic impact of R&D is given less weight. From his point of view, the weight of each category of criteria should be: scientific-technological 50%; economic 30%; social 20%.

Regarding the quantitative part of the interview, the same approach used in the case of the University expert (OTRI) was followed to draw conclusions. The Program Manager answered the two parts of the questionnaire. The initial part is intended to do an assessment of how he evaluates the Galician Plan for R&D, given his extensive knowledge and involvement with the current criteria. Since this part of the questionnaire is to draw conclusions on whether the evaluation of R&D projects is being conducted in a satisfactory manner, below (Table 5) are presented the criteria identified as poorly valued (in other words, those rated with a 1 in the questionnaire). On the other hand, as no criteria was assessed with a 5, it was assumed that in the program manager's opinion there is no criteria evaluated as excellent.

As can be seen, most of the criteria that received a score of 1 are social impact criteria. In the opinion of this expert, half of these criteria should be more valued. These criteria are related to employment issues, particularly job creation due to the applicability of the project. This is one of the two most rated criteria when having to evaluate a project, along with the scientific novelty of the project in relation to the degree of background knowledge. Both received a 5. Clearly, one can see again the importance of social impact for employees. Another criterion that would score more is the investment multiplier effect. This same criterion had been one of the leading ones for the technology centre.

Table 5
Score attributed for the criteria by Program Manager.

Criteria	Actual score	Ideal score
43. Impact on business location	1	2
21. Contribution to improving the environment	1	2
11. Staff Promotion	1	2
41. Multiplier effect of the investment	1	4
22. Creation of value added and employment with the implementation of the R?αμπ;D project	1	2
23. Value and employment creation with the implementation of the outcomes of the project	1	5
24. Execution of the project in areas with structural economic problems	1	3
25. Quality of employment generated (improving work conditions) in performing R?αμπ;D activities	1	3
26. Quality of employment generated (improving work conditions) in the external entities of those conducting R?αμπ;D	1	4
27. Increased economic and job satisfaction of employees	1	3
28. Employment of university students in R?αμπ;D activities	1	3
36. Contribution to solving social problems (unemployment)	1	4
37. Impact on political and social activities	1	2
45. Impact on the development of new businesses	1	2
38. Corporate Social Responsibility	1	1
39. Consolidation / maintenance of existing employment	1	2

4.6. Global results

In order to make a global assessment of the criteria, we focus now only on the quantitative portion of the interview. Although recognizing the limits of this approach, given that different types of organizations (e.g. companies, research institutes, trade unions) might have different opinions about the criteria to be used due to different perspectives of analyses, we show, in Table 6, the most valued criteria, performed by adding the score that each organization gave to the different criteria. From the table, one can see that in this ranking there are several criteria of social impact nature.

On the other hand, if a ranking of the least valued criteria is done, it was found that only one of them belongs to this group of social impact criteria (seeTable 7).

Overall, one can conclude from the data obtained in this study that, despite there is a clear interest in the scientific level given that we are dealing with R&D projects, among the participants of this study emerges the concern with the criteria that can measure the social impact of those projects. Among these, the most valued are: new products/services to meet unmet needs; consolidation/maintenance of existing employment; value added and employment creation resulting from the implementation of the outcome of a project; quality of the employment generated (improving working conditions) in performing R&D activities; increasing the number of jobs in R&D in universities; contribution to solving social problems (e.g. unemployment); increased level of training and staff development; corporate social responsibility; and impact on business location. As can be seen, the most valued criteria, within the category of social impact, are those related to employment.

5. Conclusions

One way for firms to maintain or increase their competitiveness is through the investment in R&D projects. Of course, the evaluation and selection of these investments is quite difficult and raises several questions. In this paper, we are interested in a particular aspect which has not been much addressed in the literature: how the social impact of R&D has been taken into account either by firms or by R&D program funding or managers. To obtain some insights into this process an empirical study was undertaken focusing on three companies of the furniture sector, two technology centres, a Technology Transfer Office from University of Vigo, three trade unions and the program manager of the Xunta de Galicia for the INCITE programme.

Table 6
Globally most valued criteria.

Criteria	Value
3. Originality of the scientific project in relation to the degree of previous knowledge	44
1. New products / services to meet unmet needs	42
42. Applicability and transferability of R?αμπ;D results	41
39. Consolidation / maintenance of existing employment	40
2. Scientific and technical level	38
21. Contribution to improving the environment	37
15. Feasibility and appropriateness of methodology	36
23. Value and employment creation with the implementation of the outcomes of the project	36
41. Multiplier effect of the investment	35
25. Quality of employment generated (improving work conditions) in performing R?αμπ;D activities	35
4. Clarity, accuracy and critical factors of the objectives	35

Table 7
Globally less valued criteria.

Criteria	Value
14. Number of Ph.D.	22
37. Impact on political and social activities	23
32. Increased wage level	25
18. Use of Galician language in the activities for which funding is requested	25
10. Adequacy of the size and composition of the research team	26
16. Degree of dedication to the project	27
20. Involvement of entities belonging to different stages of the value chain	27
40. Contribution to solving social problems (immigration)	27
6. Consistency of project objectives with the science and technology policy	28
15. Incorporation of new Ph.D. (<6 years' experience)	28

From the empirical study, it can be concluded that even if the interest in the scientific level is remarkable (it should be noted that we are referring to R&D projects) social repercussion criteria are also to be taken into account. Prime factors like the creation and maintenance of employment and the search for products and services aimed at covering unsatisfied needs are some of the most valued criteria to measure the social impact.

With regard to companies, this study has proved that there are different criteria for project selection, although not many of them are socially-based. Instead, companies are becoming more and more aware of environmental issues. On the other hand, we have found out that some firms seem to be more interested in considering social issues when it comes to assessing projects. Therefore the present research has been useful for creating social consciousness. Prime factors like search for products and services aimed at covering unsatisfied needs, staff development, contribution to environmental improvement, employment of women on R&D, social responsibility and the creation and maintenance of employment are some of the most valued criteria to measure the social impact.

To some extent surprisingly, technology centres are those that showed to be less keen on the inclusion of social criteria when assessing projects. They rather seem to be more interested in having into account scientific and economic criteria such as the multiplier effect of investment, which will indirectly imply other social criteria like employment.

Regarding the University perspective, it criticises the fact that well-established projects are not assessed by the administration, an opinion which is shared by the experienced project manager, who collaborated with us on this research. This might be helpful to assess both the current R&D plan and its benefits and also to verify if appropriate procedures are being carried out when it comes to financing. Attention should also be given to the difficulty experienced by some associations which develop R&D projects (e.g. associations of disabled people) when trying to obtain public financing.

Finally, trade unions have mainly contributed by sharing with us their view on the current Galician R&D plan and its need for change, going for public financing in order to enable R&D to work by taking into consideration social criteria. As would be expected, the most valued criteria to measure the social impact by trade unions are contribution to solving social problems (e.g. unemployment) and the creation and maintenance of employment.

We think that it is advisable to continue through several stages, such as carrying out the appropriate tasks which will enable the public administration to incorporate social criteria, when assessing applications for R&D grants, so as to represent about 20% of all the criteria to be taken into account. This would mean an increase of 15 percentage points since the current figure is lower than 5% at a regional level and only the presence of women is considered to be a social criterion. Should public administration take into account social criteria in its announcements of grants, companies and institutions would also begin to consider those when selecting the projects in which they wish to invest. In short, social benefits would be strengthened by adopting such measures.

The results of this study allowed combining information obtained from both interviews and a quantitative questionnaire, concluding on the importance of both scientific/knowledge transfer and employment impacts of R&D projects. However, the authors clearly recognize the limitations of this case study approach relying on a quantitative analysis based on Likert scale with a small set of respondents. In fact, no attempt is made to generalize or to present a final outcome for the R&D project evaluation but rather to debate the importance of going beyond the evident economic and even scientific interest of the projects, especially when these projects are supported by public funds.

Companies operate in a competitive environment where the cost and return criteria are fundamental drivers of their investments and therefore tend to value most the short term commercial benefits that R&D can bring them. Not surprisingly, the results show that aspects such as impact on political and social activities or the consistency of project objectives with the science and technology policy are not valued by most of the interviewees. This strongly demonstrates the importance of R&D support organisms to ensure that their criteria and evaluation process strongly emphasize the social and environmental impacts, acknowledge the long term benefits of these investments and guarantee the selection of projects that can contribute to the strategic objectives of the scientific and technological system of the region or country as a whole.

References

• Adler, R. (2000). Strategic investment decision appraisal techniques: the old and the new. Business Horizons, 43(6), 15-22. http://dx.doi.org/10.1016/ S0007-6813(00)80017-8
• Ares, E., Dominguez, A. P., Quintela, E., Fernández-López, F. J., & Sancho, M. (2008). Análisis del "Retorno Social" de la financiación pública de la I+D+i. Vigo: Observatório Industrial del Sector del Metal, Área de Ingenieria de los Processos de Fabricación, Universidade de Vigo. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://mcaugt.es/ documentos/0/doc2985.pdf
  » http://mcaugt.es/ documentos/0/doc2985.pdf
• Barreiro, J. B. (2012). El modelo de gestión de la superficie forestal en galicia y su repercusión en la crisis incendiaria del año 2006. Revista Galega de Economía, 21(2), 1-28. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://www.usc.es/ econo/RGE/Vol21_2/castelan/art1c.pdf
  » http://www.usc.es/ econo/RGE/Vol21_2/castelan/art1c.pdf
• Barroso, M., & Iniesta, J. (2013). Valuation of projects for power generation with renewable energy: A comparative study based on real regulatory options. Energy Policy, 55, 335-352. http://dx.doi.org/10.1016/j. enpol.2012.12.019
• Bitman, W. R., & Sharif, N. (2008) A conceptual framework for ranking R&D projects. IEEE Transactions on Engineering Management, 55(2), 267-278. http://dx.doi. org/10.1109/TEM.2008.919725
• Brownell, P., & Merchant, K. A. (1990). The budgetary and performance influences of product standardization and manufacturing process automation. Journal of Accounting Research, 28(2), 388-397. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://www.jstor.org/ stable/2491156 http://dx.doi.org/10.2307/2491156
  » http://www.jstor.org/ stable/2491156
• Carroll, A. B., & Buchholtz, A. (2009). Business and society: ethics and stakeholder management. Cincinnati: SouthWestern Publishing.
• Carvalho, M., Ferreira, P., Ares, E., & Araújo, M. (2010, March). The socio-economic return of the Research and Development (R&D) support programmes. In 2nd Conference on corporate R&D, Seville, Spain. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://hdl.handle.net/1822/19565
  » http://hdl.handle.net/1822/19565
• Chiesa, V., Frattini, F., Lazzarotti, V., & Manzini, V. (2009). Performance measurement in R&D: exploring the interplay between measurement objectives, dimensions of performance and contextual factors. R&D Management, 39(5), 487-519. http://dx.doi.org/10.1111/ j.1467-9310.2009.00554.x
• Chan, F., Chan, M., Lau, H., & Ip, R. (2001). Investment appraisal techniques for Advanced Manufacturing Technology (AMT): a literature overview. Integrated Manufacturing Systems, 12(1), 35-47. http://dx.doi. org/10.1108/09576060110361528
• Chang, H.-F., & Tzeng, G.-H. (2010). A causal decision making model for knowledge management capabilities to innovation performance in taiwan's high-tech industry. Journal of Technology Management & Innovation, 5(4), 137-146. http://dx.doi.org/10.4067/ S0718-27242010000400011
  » http://dx.doi.org/10.4067/ S0718-27242010000400011
• Duch-Brown, N., García-Quevedo, J., & Montolio, D. (2012). Assessing the assignation of public subsidies: do the experts choose the most efficient R&D projects? World Review of Science, Technology and Sustainable Development, 9(2-4), 149-168. http://dx.doi. org/10.1504/WRSTSD.2012.047686
  » http://dx.doi. org/10.1504/WRSTSD.2012.047686
• Ghasemi, E., Tajadod, M., & Naderi, A. (2011). A combined method based on integration of fuzzy analytical network process and stochastic dominance degree for R&D project selection in the electrical distribution company. In International Conference on Advances in Electrical and Electronics Engineering (ICAEE'2011), Thailand.Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://psrcentre.org/images/extraimages/1011269.pdf
  » http://psrcentre.org/images/extraimages/1011269.pdf
• Hassanzadeh, F., Collan, M., & Modarreset, M. (2012). A practical R&D selection model using fuzzy payoff method. International Journal of Advanced Manufacturing Technology, 58(1-4), 227-236. http:// dx.doi.org/10.1007/s00170-011-3364-9
  » http:// dx.doi.org/10.1007/s00170-011-3364-9
• Henig, M., & Katz, H. (1996). R&D project selection: a decision process approach. Journal of Multicriteria Decision Analysis, 5(3), 169-177. http://dx.doi. org/10.1002/(SICI)1099-1360(199609)5:3<169::AIDMCDA94>3.0.CO;2-V
  » http://dx.doi. org/10.1002/(SICI)1099-1360(199609)5:3<169::AIDMCDA94>3.0.CO;2-V
• Leite, L., Teixeira, J., & Samanez, C. (2012). Ex-ante economic assessment in incremental R&D projects: technical and development time uncertainties addressed by the real options theory. Pesquisa Operacional, 32(3), 617-641. http://dx.doi.org/10.1590/S0101-74382012005000025
  » http://dx.doi.org/10.1590/S0101-74382012005000025
• Moñux, D., Aleixandre, G., Gómez, F. J., Cáceres, S., Miguel, L. J., & Velasco, E. (2006, June). Evaluación del impacto social de proyectos de Investigación y Desarrollo tecnológico (I+D): una aplicación en el sector de las comunicaciones industriales. In I Congresso Iberoamericano de Ciencia, Tecnología, Sociedad e Innovación, México. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://www.oei.es/memoriasctsi/mesa6/m06p17.pdf
  » http://www.oei.es/memoriasctsi/mesa6/m06p17.pdf
• Nigro, G., Morreale, A., & Enea, G. (2014). Open innovation: a real option to restore value to the biopharmaceutical R&D. International Journal of Production Economics, 149, 183-193. http://dx.doi.org/10.1016/j. ijpe.2013.02.004
• Olaz, A. (2007). La entrevista en profundidad: en la encrucijada del debate metodológico de lo cuantitativo frente a lo cualitativo. In IX Congreso español de sociología: poder, cultura y civilización (Sesion 2º. Articulación metodológica: teoría y práctica), Barcelona, Spain.
• Olaz, A. (2008). La entrevista en profundidad: justificación metodológica y guía de actuación práctica. Oviedo: Septem Ediciones. PMid:17638596.
• Rip, A., & Kemp, R. (1998). Technological change. In S. Rayner & L. Malone. Human choice and climate change (Vol. 2: resources and technology). Washington: Batelle Press. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, from http://kemp. unu-merit.nl/Rip%20and%20Kemp.pdf
  » http://kemp. unu-merit.nl/Rip%20and%20Kemp.pdf
• Vandaele, N., & Decouttere, C. (2013) Sustainable R&D portfolio assessment. Decision Support Systems, 54(4), 1521-1532. http://dx.doi.org/10.1016/j. dss.2012.05.054
• Xunta de Galicia (2007). Plan Galego de investigación, desenvolvemento e innovación tecnolóxica 20062010. Retrieved in 21 November 2012, from Retrieved in 21 November 2012, fromhttp:// economiaeindustria.xunta.es/plans-idi
  » http:// economiaeindustria.xunta.es/plans-idi
• Zopounidis, C., & Doumpos, M. (2002). Multi-criteria decision aid in financial decision making: methodologies and literature review. Journal of Multi-Criteria Decision Analysis, 11(4-5), 167-186. http://dx.doi.org/10.1002/ mcda.333

Annex 1 Criteria for R&D project evaluation.

Publication Dates

• Publication in this collection
  09 Oct 2015
• Date of issue
  Oct-Dec 2015

History

• Received
  21 Nov 2012
• Accepted
  18 June 2014