Measuring innovation: the use of indicators in developing policy

The recently published ‘Handbook Of Innovation Indicators And Measurement‘ examines indicators and statistical measurement in the context of innovation.  The book’s success, according to editor Fred Gault, is driven by the fact that the contributors are practitioners in this area; they know from first-hand experience what works and what doesn’t.  In addition, this collection also presents an agenda for the development of the subject and is expected to inform discussions at the next OECD Blue Sky Indicators conference.  

In answering the questions that follow, Fred provides further details about his area of research and explains the significance of innovation indicators in developing policy.

Innovation is the solution to our problems but …
Innovation policies target jobs and growth as their preferred outcome. In developing countries there is interest in inclusive innovation, where the included are people with characteristics such as the poor, the sick, the illiterate, or members of minority groups. The question that arises is how do people know that an innovation policy, once implemented, has created jobs, growth or inclusion. The answer starts with indicators and their use in the policy process. Indicators matter.

How do we talk about innovation?
‘Innovation’ is an overused word but what does it mean? Everyone has their own definition, but for measurement purposes there is only one and it is found in the OECD/Eurostat Oslo Manual. Paraphrasing the Oslo Manual definition, innovation is the putting of a new or significantly improved good or service on the market or finding a better way of getting it there. Novelty and ‘market’ are key and ‘market’ limits this definition to firms; only firms can innovate.

While this definition is restrictive, it has guided statistical surveys, such as the European Union Community Innovation Survey (CIS), and similar surveys, for over 20 years. The definition and the surveys have established a language that has shaped the innovation discourse and the definition and the language are still evolving.

Can innovation be measured?
The definition of innovation is implemented in surveys of businesses which produce data, such as whether or not a business innovates, which are used to populate statistics, such as the propensity for businesses to innovate, and the statistics may be used as indicators, on their own or with other statistics. Other statistics provide information on the businesses that are or are not innovative. Innovation surveys of the CIS type ask about behaviour of the firm. Did it put a new product on the market? Did it find a better way of getting it there? Yes or No. This measurement does not include whether the innovation is good or bad, inclusive or exclusive, or with long term impacts or not. Innovation can be measured.

How are indicators used?
Indicators of innovation can show regional differences, industry differences, and differences in the skill levels of workers in firms that innovate. These differences may attract policy interventions, and trends, observed from data resulting from repeated surveys of innovation, may be used to show the progress of the intervention. If a causal link is needed between a policy intervention and an outcome, panel data are required. These result from the surveying of the same firms in a panel over a number of years. The indicators resulting from surveys are published in scoreboards and provide a basis for international comparisons, benchmarking and target setting. The approach to policy and the use of indicators in support of policy differs from country to country and the Handbook provides examples from Finland, Japan and the US.

What’s new?
The Oslo definition of innovation applies only to firms, but there are public institutions that introduce new or significantly improved services (or goods), not to the market but to potential users. They also find better ways of doing this by developing, and implementing, more effective policies or organizing themselves differently. There is a growing literature on public sector ‘innovation’ and the Handbook discusses how this could fit with the approach of the Oslo Manual.

Then, there are consumers that change goods or services for their own benefit and then share the knowledge with peer groups or communities of practice, or present it to a firm that produces such products or, they may become entrepreneurs and start a firm. These people are not innovators according to the Oslo Manual, but, as with the public sector, there is a discussion of how to bring them into a broader picture of innovation, without losing the precision of the original definition.

Working within the Oslo definition, there is work on how to see weak signals of innovation resulting from the use of new technologies, such a bio and nano-technologies and new materials. The challenge is to find these emerging activities so that they can be tracked and their outcomes and impacts can be measured. There are those who are not content to find what is out there now. They want to see what is going to happen rather than what is happening. This work on foresight shapes the development of indicators.

Then there is social ‘innovation’ where communities do new or significantly improved things to improve their welfare, or find better ways of doing those things. This is an important area for future work, as governments transfer more public services to the voluntary sector and better indicators are required to understand, promote and evaluate social ‘innovation’.

Is anything going to happen?
The Handbook is written by people who do what they write about and who want to transfer their knowledge to the reader but, in addition, the Handbook is an agenda for the development of the subject. It is expected to inform the discussion around the next revision of the Oslo Manual and the next OECD Blue Sky Indicators conference expected in 2016. Blue Sky conferences happen once a decade and the outcomes of the 2006 conference are still influencing the agenda.

More broadly, there are innovation surveys and policy activities in developing countries, as innovation is important everywhere.  The Handbook supports learning by doing and using innovation indicators in policies for development.

Fred Gault is a Professorial Fellow at UNU-MERIT in the Netherlands and a Professor Extraordinaire at the Tshwane University of Technology in South Africa. A former Chair of the OECD Working Party of National Experts on Science and Technology Indicators (NESTI), he chairs the NESTI Advisory Board. For many years he was responsible for science, technology and innovation indicators at Statistics Canada.

This article originally appeared on the Elgarblog.