Evaluating R&D with "First Bounce-Last Bounce" framework: application to the European Space Agency demonstrates the usefulness of a new methodology for assessing R&D programs

Research-Technology Management, Jan-Feb, 2006 by Marco Guglielmi, Stephane Lascar, Vittorio Mastrocola, Edmund Williams

The European Space Agency (ESA) is an intergovernmental organization with the mandate of promoting and supporting cooperation among its member states in space research, technology development and applications. To fulfill this mandate, ESA finances every year a large number of R&D activities. The evaluation of the effectiveness of the R&D effort is one of the key elements of the ESA Space Technology Strategy. In this context, a new framework called the "First Bounce-Last Bounce" Framework has been developed.

In this paper, we first describe the First Bounce-Last Bounce Framework. We then discuss the application of the Framework to a real case thereby showing its effectiveness in the ESA context. Finally, the applicability of the Framework developed to other fields is discussed.

Literature Survey

To base the development of the R&D evaluation framework on a robust theoretical background, we first performed a survey of the state of the art. The survey (next page) enabled the identification of:

* The most applied methods and techniques to evaluate technology R&D.

* The most important parameters to assess the effect of R&D activities

The process, which extends from Research to Commercialization is commonly referred to as the "Innovation Continuum" and encompasses all the main stages of the product development (1). The literature survey has shown that many attempts have been made to measure the effectiveness of the R&D activities. Some researchers (1) have tried to establish a direct link between original ideas and the development of a new product, while others have tried to link R&D spending and revenue growth (2). Others have proposed breaking the innovation continuum into a number of intermediate stages (segmentation) with a subsequent assessment of the intermediate outputs of each stage (1). Finally, an attempt has also been made to identify direct and indirect effects of R&D activities (3).

Establishing a direct link between R&D and sales or revenues is, of course, very attractive. Unfortunately, this approach is not always feasible. An effective direct link can, in fact, be established only if a very short time and few players are involved in the transformation of the idea into a product. A certain degree of complexity can indeed be accommodated by segmentation. However, when ten or more years are needed to transform an idea into a product, and a large number of contributors are involved, another approach must be developed.

First Bounce-Last Bounce Framework

One way of visualizing the process that a generic initial idea follows during its evolution toward product commercialization is through what we call the "bounce" concept. It is possible to say that the original idea "bounces" through the different stages of the innovation process from the initial conception to the final stage: the sale of a new product in the market. Each time that a bounce occurs (an event or R&D action), a number of outputs are generated. These outputs then become some of the inputs for the next bounce for the product or technology under development, but could also become inputs for other actions not related to the original idea. Similarly, the next R&D action for the product under development could, in principle, benefit from the outputs of an unrelated activity. The overall process, thus, becomes non-linear (see Figure 1).

[FIGURE 1 OMITTED]

The long time involved in transforming the space R&D outputs into a product ready for market makes even more complex the issue of the non-linearity associated with the innovation process. This non-linearity can be by-passed with the proposed Framework.

The first step in developing the framework is to consider the environment in which ESA operates as a single "system," composed of all the European companies and research centers working in space-related activities (the European space industry landscape). The next step is to identify the ESA R&D-funded activities as the "inputs" to this system. To complete the picture, the "output" of this system is the global performance of the European space industry in terms of its contribution to fulfilling the ESA mission objectives, and in terms of the general worldwide competitive position of its products.

The analysis of the performance of the system thus defined can then be carried out using the following three elements (the First Bounce--Last Bounce framework):

1. The First Bounce survey: aimed at monitoring the generation of direct outputs from the ESA funded-R&D contracts (e.g., referred technical publications, patents, know-how, first revenues, etc.)

2. The Last Bounce survey: aimed at identifying and collecting data from the "best performing" companies in the European space industry.

3. The analysis of the data collected to identify the key enabling factors at the origin of the outstanding "performance."

Validation

To validate the applicability of the framework developed, we carried out the two surveys on a restricted but relevant "population." For the First Bounce survey, we chose a suitable set of past R&D contracts, while for the Last Bounce survey we limited the scope to the European satellite equipment manufacturing industry. Both surveys have been carried out with the help of questionnaires.