Transition Management: Case Study of an Energy Efficiency Technology Roadmap in Turkey

This paper integrates several streams of literature in transition management and proposes a holistic framework for its application in policy-making. Separate fields of study, such as motors of change and strategic intelligence tools, are unified in a single analytical process. The process involves five steps that may be repeated until a desired policy objective is achieved. The pilot, integrated technology roadmap process that has been launched in Turkey is analyzed as a case study. The Energy Efficiency Technology Roadmap has been completed with the participation of over 160 experts in 5 different stages. It involved the collection of over 349 Delfi statements, their consolidation for a Delfi survey with 16 statements, the analysis of the results, a focal group meeting to develop roadmaps for the 7 selected goals, and the consultation of the roadmaps to the sector. The paper concludes that an integrated technology roadmap process, as described in the pilot case study, provides an advanced version of transition management, which is needed to mobilize research, development, and innovation for sustainable development.


INTRODUCTION
Transition management involves the steering of complex societal systems towards the goal of sustainable development [1,2], which requires the decoupling of economic growth from environmental pressure [3]. In so doing, transition management identifies the steps that may be taken from a governance perspective to transform the complex systems of society into more sustainable future states. This requires a "complex systems thinking" to determine the steps that should be taken between the present and the future.
To aid in easing the conceptualization of the framework of transition management, three spheres are identified. These are spheres that involve strategic (i.e. what should be achieved), tactical (i.e. how to reach there), and operational (i.e. how to do it) efforts, which are necessary to reach a desired, normative goal of a future that is expected to be more sustainable than the present [4]. When taken as a whole, these basic spheres distinguish the different kinds of governance efforts that are needed to ensure that more strategic perspectives are not left unaccompanied by the other perspectives, i.e. tactical and operational spheres, which are also needed to make such a transition possible.
Presently, there are two versions of transition management. Transition management that is termed version 2 (v2) involves a greater emphasis on empowering the actors of complex systems to act in the direction of sustainable development (i.e. greater emphasis on tactical and operational spheres) over an emphasis on foresight for sustainable need to customize technology roadmaps according to the particular roadmapping purpose [13].
Czaplicka-Kolarz et al. present a technology foresight initiative for a vision of energy sector development in Poland with the aim of providing sectoral development directions in a time horizon up to 2030 and identifying key energy technologies of strategic importance [14]. Rogut et al. overview methods of foresight that can be used as tools to elaborate plans for the sustainable management of water, energy, the environment, and society [15]. These include the use of Delphi surveys, listing of critical technologies, technology mapping, analysis of trends, wild cards, and scenarios [15]. Celiktas et al. overview the results of a two-round Delphi survey for the renewable energy sector in Turkey [16]. Similarly, Preisler et al. present the process of a technology roadmap for solar thermal cooling in Austria [17]. Other studies include Keller at al. that apply a Delphi survey to the ICT sector [18] and Rödel et al. that detail a roadmap for advanced ceramics [19].
The process of all of these efforts share commonalities in the kinds of foresight methods that are used, most clearly the Delphi method to provide policy direction to roadmapping. On the other hand, Jeffrey et al. evaluate the success of roadmaps based on whether its objectives have been translated into actions or policies [20]. Yet other studies describe roadmaps for carbon capture [21], the power sector [22], the renewable energy sector [23], and a low carbon society [24]. Sasaki et al. describe a stakeholder's meeting, technology assessment, and consensus building to increase energy efficiency in Lao [25].
Carvalho et al. have combined bibliometrics, content analysis, and semantic analysis in studying the literature on technology roadmapping [26]. The bibliometric map of the intensity of the relations among the elements addressed in the literature indicate that papers focusing on the topic of "technology roadmapping" most often also focus on innovation, disruptive innovation, and science [26]. Relatively, topics that are less often co-present in the same paper are technology management, industry, and evolution [26].
In another aspect, McDowall et al. review the literature that involves concepts for a hydrogen future and find that these have involved forecasts, exploratory scenarios, technical scenarios, visions, roadmaps, backcasts and pathways, and roadmaps [27]. The same authors claim that there has been little cross-over between "transition scenarios" theory and technology roadmaps [28]. As a result, although the two policy frameworks have similarities in purpose, the framework of transition management and the process of technology roadmapping have not been completely integrated in the literature [28].

Contributions to the literature
In line with these trends, this paper integrates several streams of literature in transition management (including transition scenarios), technology roadmapping, and innovation system policy and proposes a holistic framework for their application. Separate fields of study that are integrated from the innovation policy literature into this framework are system failures [29,30], systemic policy instruments [31], functional dynamics [32,33], motors of change [33,34], and strategic intelligence tools [35][36][37][38]. Figure 1 provides the span of these concepts as well as the three separate groups of literature from which concepts are unified into a single analytical process. This unified process has five repeatable steps to solve an unaddressed issue in the literature.
The process of developing an Energy Efficiency Technology Roadmap in Turkey is analyzed as a case study. In this pilot initiative, experts from the energy efficiency field were brought together with a systematic methodology to indentify needs, technologies, qualitative metrics, development gaps, prominent tactics to fill the gaps, short term and long term milestones, and the target audience. These attributes of the methodology are supported by the collection of Delfi statements, their consolidation into a Delfi survey, the analysis of the results, a focal group meeting to develop roadmaps for the selected goals, and the consultation of the technology roadmaps to the sector.
The scientific novelty of the paper arises from the integration transition management, technology roadmapping, and innovation policy concepts into a single analytical process and the application of this process to a pilot case study. Lessons from this pilot case study for an advanced version of transition management are further discussed in this paper.

HOLISTIC FRAMEWORK FOR TRANSITION MANAGEMENT
By nature, transition management provides a framework for the spheres of governance that must be activated in the process of reaching a more sustainable future, i.e. strategic, tactical, and operational spheres. The sphere of monitoring the progress made in the steps that are taken to reach the desired future state is also included as a complementary sphere [1,2]. While the literature on transition management has been influential in various fields, including governance, innovation system policy, and technological foresight, the studies that have branched out or have been related to transition management have been widely dispersed. A glimpse at the Transition Network [39] will be sufficient to give an example of the diversity and dispersion of cases that are associated with transition management.
From a more theoretical perspective with a scope that is focused on innovation system policy, there are a handful of concepts that are in fact related to transition management but have not been directly integrated into its three main spheres of governance (see Figure 1). These concepts that are introduced separately in the literature with no effort to unify them with the spheres of transition management include system failures [29,30], systemic policy instruments [31], functional dynamics of innovation systems [32,33], motors of change [33,34], and strategic intelligence tools, such as backcasting and foresight [35][36][37][38]. In an effort to unify these concepts in an advanced version for transition management, each of these concepts will be introduced in turn before a single analytical process is proposed.

System failures
An innovation system can be seen as an interconnected "ecosystem" with actors from public and private institutions, including the industry, university, and public research institutes. This ecosystem-like nature requires the synergetic working of all of its components, which also makes it vulnerable to failures that can affect the entire system.
It is known that any system is only as strong as its weakest component [40,41]. In this respect, it becomes critical to take a look at system failures, which are defined as failures that impact the working of an entire system [30]. In the innovation systems literature, these failures can be categorized into at least seven kinds, including learning and capacity failures, transition failures, and collaboration failures [29]. For example, the lack of entrepreneurial activities, which are important for radical innovations in clean technology, may be due to a lack of skills in entrepreneurial capacity. On the other hand, the lack of actors in a specific sector that relates to clean technology may be due to a "lock-in" [42] on incumbent technology, which is also indicative of a transition failure [29]. Yet another example is when actors of a given technology are hesitant to collaborate with other actors to establish broader collaboration networks [29]. Failures such as these will affect the working of entire innovation systems and as a result, have ramifications on the future projectiles of technological progress.

Systemic policy instruments
It is advisable that systemic policy instruments are adopted to address the system failures that may be pinpointed in the functioning of innovation systems. Systemic policy instruments are defined as those support mechanisms and tools that can be used to improve the performance of innovation systems [31]. For better ease of comparison, such tools have been categorized into groups, including tools that prevent lock-in (e.g. strategic niche management), engage the actors in interaction (e.g. collaborative programs, sectoral mobility scholarships), and provide strategic knowledge (e.g. foresight studies, trend analysis, and portfolio management) [31]. While it is possible that one kind of policy instrument can be categorized into two groups at the same time (e.g. strategic public procurement [43]), a view of the types of systemic policy instruments are helpful in scanning the possibilities to identify a potential solution to address a system failure.

Functional dynamics
Innovation systems differ from technology to technology, sector to sector, and most evidently, from country to country. However, it is possible to identify the basic functions or activities that all innovation systems need to fulfil in order to define a "well-functioning" innovation system [32]. For this very reason, six different functions have been identified for innovation systems. These six functions can be summarized as the promotion of entrepreneurial activities and entrepreneurial learning, knowledge development (in all performing sectors), knowledge diffusion (collaboration networks, etc.), guidance of search and selection (e.g. sectoral strategies, thematic calls), market formation (standards and regulations), and the development and mobilization of resources (human resources, research infrastructure, and financial resources) [32][33]. Within a systems view, the lack or weakness of one function will affect the performance of the system as a whole.

Motors of change
One of the recent developments in the field of innovation policy has been the view of "motors of change" [34] based on the synergetic working of more than one function to create a loop of feedbacks that accelerate technological change. Four main motors have been identified in the literature, namely the science push motor, the entrepreneurship motor, the system building motor, and the market driven motor [34]. Each of these motors can further be related to the "S-curve" of technological progress that starts with a build-up phase, follows with a take-off, and then reaches maturity before another radical innovation challenges the decline of the technology. Therefore, the four main motors can also be seen as the phases of technological progress from the emergence of a new technology to its eventual growth out from a niche market before coming to a stage of maturity in which it reaches prominence over the incumbent technology. The main driving function of each motor are described below along with some real cases.
Science push motor. This motor is relevant for the development of a new, emerging technology, especially when entrepreneurial forces are missing or weak to allow the system to progress to the next phases. Its main driving function is the function of "guidance of search and selection" that triggers knowledge development and knowledge diffusion [34]. It can be stated that technological progress benefits from a "science and technology push motor" when, for example, a targeted support program produces a feedback loop where the actors become active in developing and succeeding in demonstrating a new generation of technology. This requires the coupled triggers of providing both guidance and mobilizing resources to a targeted area. A case of this motor has been analyzed by the author with the sequence of events mapped out to the relevant dynamics after the launch of the Fuel Cell Development Program in the Netherlands [44]. This program was useful for developing molten carbonate and solid oxide fuel cells around the 1990's in the Netherlands. This was followed by the launch of a joint venture in the sector.
Entrepreneurship driven motor. A sustainable technology innovation may represent a case of an entrepreneurship driven motor when a series of technological developments are triggered by the presence of entrepreneurial activities. As a result, in this motor, the function of entrepreneurial learning is seen to be the triggering function. At the same time, the other functions must be engaged in providing feedbacks to entrepreneurial activities in order to allow such activities to sustain themselves. While the GAVE program in the Netherlands has been stated to be such a case of an entrepreneurial motor [34], the series of events have been mapped out to the functions of this motor by the author. This case was useful in allowing the development of biofuels in the Netherlands, which was initiated and driven by private actors, to continue in the direction of newer second generation biofuels, such as lignocellulosic ethanol. Since this motor involves the trigger of entrepreneurial activities, advocacy support and gaining legitimacy is also important.
System building motor. Perhaps the motor that requires the most dedicated effort with a certain background in the previous motors beforehand is the system building motor. This is the motor in which the activities of the system actors lead to the "scaling-up" of a given technology from a niche market to an increase in the market share of a technology by at least 80% [45]. As a result, this intense effort requires the collaboration of the system actors in well-defined networks that provide feedback into knowledge development and the provision of guidance to the sector. For this reason, the two trigger points of this motor are entrepreneurial activities and knowledge diffusion [44]. The main feedback loops in this motor have been mapped out by the author to the series of functions based on the sequence of events around the launch and implementation of the Biofuel Research Program in Sweden. This program was influential in allowing 6% of the bus fleet of Sweden to run on biofuels and to let the city of Linköping have the largest biofuel bus fleet in the world [44]. In addition to targeted R&D programs, pilot demonstration projects and tax benefits are also critical for this motor based on the presence of strong sectoral networks.
Market driven motor. The final phase that a technological innovation system can reach is the phase of a market driven motor. In this motor, a technology that has reached a certain level of maturity in competing with an incumbent technology is able to sustain the kind of activities that are necessary for its continued presence in the market. For this reason, the main trigger of this motor is identified to be the function of market formation [44]. In this context, the existing set of signals from the market, including standards and regulatory policies, are sufficient to support the future development of the technology [34]. However, reaching this phase is not an easy climb as demonstrated by the case of wind turbine development in Denmark. Based on a description of this case in the literature [34] and additional research by the author, this case has been traced to have gone fully through two motors before reaching the market driven phase [44]. The market driven phase started in the early 1990's when various financial incentives for wind turbines were removed in Denmark yet the sector was able to grow to become one of the largest export sectors of the country. Wind farm cooperatives and sectoral networks were also beneficial in making such a success story possible. The Mega Wind Partnership is yet another recent development [44].

Strategic intelligence tools
In many of the motors that have been described above, the function of the "guidance of search and selection" has included the usage of various strategic intelligence tools. While it is a kind of systemic policy instrument, which can include foresight studies, trend analysis, and portfolio management, there is more to be said about these kinds of tools. First of all, in the context of transition management, the question of defining the desired future state, by default, requires the use of some kind of foresight studies.
Foresight is not the same as forecast, which involves the extrapolation of present trends into a certain time into the future [36]. The use of other strategic tools, such as trend analysis based on patent trends, could be a kind of example of a forecast (e.g. whether a technology is in a downward or upward trend). In contrast, foresight involves a more trend-changing outlook to present trends to be able to define more sustainable future states. Since the foresight of a desirable future is not sufficient to define the steps to reach there, it is related to backcasting [36]. Backcasting is literally what the words stand forit requires that a view of a desirable future is adopted from which steps are worked backwards to identify the steps that should be taken at the present to reach the desired future state. This is in contrast to common practise where often incremental steps are taken from the present to the future.

Unification of concepts in a holistic framework
The five main concepts as given above, namely system failures, systemic policy instruments, functional dynamics, motors of change, and strategic intelligence tools all seem to be related to transition management. Each of these concepts either aim to identify or fix the functioning of innovation systems, determine feedbacks that create synergy between the functions, or aim to assist not only in providing pure speed and acceleration but also purposeful direction to technological progress. More recently, the term of "sustainability-oriented innovation systems (SoIS)" has also been put forth to distinguish innovation systems that are directed to the aim of sustainable technology innovations [46]. On the other hand, while all of these concepts seem to be related in purpose, they are in fact currently unrelated to the three main spheres of governance in transition management. For this reason, as a contribution to the literature, each of the five main concepts will briefly be related to the strategic, tactical, and operational spheres of governance in transition management.
Strategic sphere of transition management. This sphere of governance requires a multi-actor involvement in answering the question, "what should be achieved?" As a result, it requires the use of systemic policy instruments, such as strategic intelligence tools, to develop a foresight into the future. In so doing, it should address another systemic failure that may be affecting the innovation system. This is a possible lack of balance between investments in technology that have nearer term returns versus more radical technology with longer term benefits, e.g. a mismatch failure in incremental versus radical innovation [29]. The answer to the question should also focus on the motor of change that should be the ultimate goal, e.g. the market driven motor.
Tactical sphere of transition management. Once a view of the future is set-out, the tactical sphere of transition management requires an answer to the question, "how to reach there?" For this reason, there is an acceptance of a gap between the present state and the desired future state. The tactical dimension of this acceptance should then be a determination of a method to fill-in the gap. While the backcasting approach has been used for normative goals, it should also be utilized in view of the motors of change. Any systemic failure that might thus be inhibiting progress from one phase to another possible phrase of the motors would need to be identified along with the right set of policy tools to place the correct motor into accelerated motion.
Operational sphere of transition management. After the identification of tactical needs to make progress towards a desired state in the future, the operational stage of activities enter into play. This requires that the right set of policy tools is implemented to activate the motor that is nearest to the present state, i.e. a fitting match between tools and the needs of a potential motor. The operational stage should also include an element of monitoring to ensure that the policies that are set into action are in reality producing the expected impacts.

Sequence of five steps in a single analytical process
Based on a harmonized view of the concepts in the literature, an advanced version of transition management is proposed to involve five steps as shown in Figure 2. The five steps are, putting forth the goal, selecting the applicable motor of change, identifying systemic failures that are specific to the motor, deploying policy instruments that are needed to activate the motor, and repeating the steps based on progress towards the goal.

CASE STUDY OF AN ENERGY EFFICIENCY TECHNOLOGY ROADMAP
In Turkey, an integrated technology roadmap process has been completed in 2013 based on an understanding of the advanced version of transition management as described in this paper. It was decided by the Scientific and Technological Research Council of Turkey (TUBITAK) that technology roadmaps should be developed in pilot topics of priority. Based on this decision, the process of the preparation of the "Energy Efficiency Technology Roadmap" has been coordinated by the author. This process involved the participation of over 160 experts in 7 different stages. These stages started with the collection of over 349 statements, their final consolidation by an expert group, a Delfi survey with 16 statements, the analysis of the results, a focal group meeting to develop roadmaps for 7 selected goals, and the consultation of the roadmaps to the sector. Research capacity and patent analysis was further carried out to support the transition to a more sustainable energy future and pilot inhibitor surveys were conducted. The stages of the technology roadmapping process for the pilot topic of energy efficiency and their relation to transition management are given in the descriptions that follow.

Strategic features of the technology roadmapping process
As a foresight tool, a Delfi survey depends on the inputs of a broad base of stakeholders in determining objectives for the future. Like the name "Delfi" that comes from a mythological feature who was acknowledged for wisdom and foresight, the Delfi survey method is based on the belief that the combined view of a broad base of stakeholders will represent the best possibility in determining goals for the future [47]. In the technology roadmapping process for energy efficiency, a quasi two-stage Delfi survey was conducted.
This involved the collection of potential Delfi statements with an open questionnaire method, the consolidation of all of the statements that were collected to select a limited number of Delfi statements for the Delfi survey, and the evaluation of the results by a focal group. The reason that the term "quasi" two-stage Delfi survey is used is due to the fact that the results of the first round of the Delfi survey were not shared with the entire participating group before a second round with the same group was conducted. Instead, the second round was left to the evaluations of the focal group, who analyzed the results of the first round of the survey before their final selection.
Collection of potential Delfi statements. Studies that were conducted by TUBITAK identified a pool of experts in energy efficiency to whom open-ended questionnaires were sent to receive their suggestion for 5 potential Delfi statements. To eliminate the possibility that some experts could have been left unidentified, a web and Twitter announcement were also made at the same time. The pool of experts included project coordinators in universities and the private sector, international researchers with connections to Turkey who are living aboard, young entrepreneurs, and representatives of sectoral organizations in the field of energy efficiency in Turkey. In total, 349 complete statements were collected from 96 experts in the field of energy efficiency across various sectors from young entrepreneurs to more mature project coordinators to sectoral representatives. In receiving the inputs, it was requested that the Delfi statements should specify a specific technology and/or technology application, have measurable values, and are time-bounded, i.e. be "smart" statements.

Consolidation of statements for the Delfi survey.
It is clear that a Delfi survey cannot include all of the 349 statements that were received by the open questionnaire method. For this reason, a step to limit the amount of potential Delfi statements was needed. This was done in two ways; first, all of the statements were subjugated to the criteria to be a Delfi statement (e.g. whether it pointed to specific technology, etc.). It was found that some of the statements involved suggestions for increased investment without specifying any aspect of technology development. The second was to invite a sub-group of energy efficiency experts to consolidate the Delfi statements. To ease the second step, a total of 135 statements that passed the Delfi statement criteria were organized into Mindmap figures based on a taxonomy that was developed for energy efficiency technology. These included groupings into efficient building technology and its sub-categories (e.g. new insulation materials), industrial equipment, and energy efficiency in transportation (e.g. improved combustion engines). The group of energy efficiency in the energy system was included for suggestions involving energy storage and smart grids. The distribution of the number of Delfi statements per each branch in the Mindmap figures was also provided.
The aim of the meeting with the experts was to reach consensus on the limited number of statements that should be included in the Delfi survey. In the meeting, the experts expressed their views, voted on the best possible statements, and discussed ways of combining and consolidating the statements with the top votes. At the end of the meeting, consensus was reached by the experts to include 16 statements in the Delfi survey.
Implementation of the Delfi survey. In the Delfi survey, each of the 16 statements was evaluated against specific questions with selectable answers that were used to construct feasibility and importance indices. Such indices had also been used by TUBITAK in a previous foresight project called Vision 2023 [48]. A web-based, pin-coded survey was conducted using a questionnaire on LimeSurvey. Before giving marks to the statements, the experts were asked to rank their level of expertise in relation to each statement since an expert in energy efficiency may not necessary be an expert in each related field of technology. The level of expertise for each statement was used to weight the rest of the marks of that statement, which is a necessary component of a Delfi survey [47]. In the following questions, each statement was evaluated based on the level of innovative capability in firms, the presence of sufficient R&D actors, the stock of scientific knowledge in the field, and the status of R&D infrastructure. The answers to these questions were used to construct the feasibility index with different weights that were given depending on whether the current activity in relation to the statement was in the development or industrial validation stages. The importance index was based on the potential impact of the statement on improving energy intensity, environmental benefits, such as reducing CO 2 emissions, societal benefit, and increasing economic welfare.
Evaluation of the survey results by a focal group. The design of the Delfi survey enabled each statement to be placed on a quadrant with axes for the values of the feasibility and importance indices. For example, a statement that may have a low level of feasibility may have been found to have a very high level of importance for a sustainable energy future. In this case, it may be justified to receive additional investment to increase R&D capacity to enable that statement to be realized. On the other hand, a statement with a high level of feasibility may not have a very high impact to improve the energy intensity of the economy, reduce CO 2 emissions, or improve socio-environment benefits. In this case, it may not be necessary to make further investments in the technology that is identified in that statement.
These kinds of evaluations based on the quadrant layout were done in a focal group that was convened by TUBITAK to evaluate the outcomes of the Delfi survey. Prior to the meeting of the focal group, the results of the Delfi survey were also sent to all private sector representatives to allow them to select the 3 most important Delfi statements after reviewing the survey results. With these inputs, the focal group of 10 leading experts in the field of energy efficiency selected the most important 7 Delfi statements. The futuristic statements that would be the focus of R&D efforts in Turkey for the next 10 years were thus selected, completing the strategic sphere.

Tactical features of the technology roadmapping process
In preparatory studies that led to the launch of the technology roadmapping process for energy efficiency, there were related studies that indicated the systemic failures in the energy R&D sector. One of these studies included the carrying out of a pilot survey to identify "R&D inhibitors" in the energy sector. For this, a total of 6 pilot questionnaires were conducted with leading R&D managers in the energy sector, including those of firms that had produced electric appliances with A+++ energy labels, thin-film photovoltaic technology, hydrogen fuel cells, energy storage batteries, and leading automotive designs. These firms were located in the provinces of Ankara, Istanbul, and Bursa in Turkey. The design of the questionnaire encompassed the scope of possible systemic failures but worded these potential areas of improvement in simple terms without stating their exact term in the literature. For example, some of the questions asked whether there was a lack of entrepreneurial actors in the sector (energy efficiency, renewable energy etc.), whether there were enough private sector actors but lack of R&D capability, whether the actors were eager to form collaboration networks, and the nature of collaboration with universities and public research institutes.
After the results of the pilot questionnaires with the R&D managers were analyzed, it was seen that the energy sector as a whole in Turkey was in need of advancing targeted R&D capacity. Especially in the energy efficiency sector, it was stated that there was an important level of R&D capacity but it was dispersed throughout different actors in Turkey without any dedicated policy focus to create synergy across them, which limited the ability to develop radical and innovative energy technologies. With this in mind, the technology roadmapping process was seen to be particularly important to set direction for R&D activities in the field of energy efficiency. This also acknowledged that a version of the "science push motor" would be necessary before it was possible to advance to the entrepreneurial or system building motors.
Determination of milestones to reach the selected Delfi statements. With 7 Delfi statements having been selected based on the survey results, the same focal group was asked to determine the milestones that would be needed to realize the goals within the statements. In this task, the working groups of the focal group were asked to determine the products and the specifications of the products that would need to be developed to allow the future statement to be realized. This task was closest to the type of product planning roadmap that was identified in the literature by Phaal [12].
An important aspect of determining the milestones involved the task of identifying the technological gaps, i.e. the gap between the present and the desired level of technology that should be developed to be able to produce products that satisfy the exact specifications. The necessities to fill these technological gaps were then transformed into milestones so that it would be possible to develop the capability to produce all of the products that would be needed to fulfil the Delfi statements. This process was repeated by the working groups for all of the statements for which they were responsible. The results were then shared with the focal group and a discussion was conducted by a moderator from the sector. This step for determining the milestones was a critical aspect of the tactical phase in setting the science push motor of change into motion. It answered the question "how to reach there?" by also pinpointing the technologies that should be developed in the science push motor of change.
Consultation of the technology roadmaps to the entire sector. The milestones that were put forth for each of the 7 Delfi statements were opened for comments from all of the participants that had contributed to the technology roadmapping process from the beginning. The final outcomes of the focal groups, which also included timeframes for each milestone, were sent to over 160 experts that had been involved in the technology roadmapping process in various ways. The inputs of this step were again coordinated by TUBITAK and consulted to the focal group experts as necessary.

Operational features of the technology roadmapping process
The support mechanisms that will be used to realize the technology roadmap is as important as both of the two previous features combined. This is because without the transfer of financial resources to realize the strategic goals and milestones of the technology roadmap, it is not even possible to discuss about the possibility of having the Delfi statements realized. The fact that there were two support mechanisms that were waiting for the results of the technology roadmap was an advantage in this respect.
These mechanisms were the TUBITAK 1003 and 1511 coded programs for the support of R&D activities in prioritized areas, both of which were call-based. These mechanisms had been in effect since 2012 but their calls would be based on technology roadmaps for the first time starting with the Energy Efficiency Technology Roadmap. Since the opening of calls based on the milestones of a roadmap was a new process at TUBITAK, two separate meetings with the officials who were responsible from the programs were organized. This ensured that the internal process of preparing the calls proceeded in line with the milestones of the technology roadmaps. Currently, 12 calls have been opened based on the Roadmap.
Coordination meetings to create integrity in the calls. A tentative call program based on the milestones was drafted and presented to the officials who are responsible from the two call-based programs of TUBITAK, 1003 and 1511. These programs differ in their target group, with 1003 being geared towards university and public research institute actors and 1511 being geared towards private sector actors. Both, however, incentivize collaboration between university and the industry with financial support.
Opening of calls based on the milestones of the roadmap. Each of the programs followed their internal procedures for drafting the calls based on consultation groups that were formed from representative experts in the sector. These procedures have been established by the relevant by-laws of both of the TUBITAK programs.

Integration of all three governance spheres for an energy transition
As described above, features of the technology roadmapping process are integrated with the key features in the strategic, tactical, and operational spheres of the advanced version of transition management. As summarized in Figure 3, the spheres start with a strategic view at a time in the future (t f ) and work back into the tactical (t f-x1 ) and operational (t f-x2 ) spheres. A key bridging factor between the present and the future is the planning that is involved in activating the motor of change and implementing the needed milestones to close the technological gaps. The inset of the figure includes a sampling of the Delphi analysis results for a given statement (expertise weighted results of the feasibility and importance indices) and pictures from the meetings.

DISCUSSION OF RESULTS OF THE TECHNOLOGY ROADMAP
The described process of preparing and initiating the implementation of the Energy Efficiency Technology Roadmap has satisfied four of the five steps of the advanced version of transition management as proposed in this paper.
First, the step of putting forth the goals for the innovation system to which to aim towards has been satisfied by the carrying out of the Delfi survey. Most of these statements target the development of technology with some also giving targets for the diffusion of technology to reach a certain percentage of energy saving in the sectors.
In addition, the outcomes of the discussions with R&D managers in pilot surveys to locate R&D inhibitors and the task of identifying technological gaps by the focal group were used to validate the selection of the applicable motor of change as the "science push motor." This included an understanding of the systemic failures that can affect this motor of change, such as a lack of balance between investing in technology with near term returns versus technology with longer term benefits. As a result, the second and third steps of the proposed framework have further been satisfied.
The fourth step, which is the step of deploying policy instruments that are needed to activate the motor, is met with the opening of calls in the call-based programs of TUBITAK. Due to the very recent completion of the fourth step, the fifth step has not yet been initiated, which requires the repetition of the steps based on progress towards the goal. This is planned for the near future with evaluation and concertation meetings. However, an interim step has been taken with the review of the Roadmap in the Energy Efficiency Panel of the Second Congress for International Turkish Scientists. It is envisioned that such evaluation mediums will be useful to evaluate progress towards the Delfi statements, either from a technological trajectory, capacity point of view, or both. This is also needed for realizing the progress of phases in the series of the motors of change from the science push to the market driven motors as indicated within Figure 2 (first two motors).

General overview of energy efficiency goals in the roadmap
The recent Energy Efficiency Technology Roadmap involves seven future oriented, technological goals that will improve energy efficiency in buildings and the industry. These goals can be classified based on whether they represent supply side technologies for the more efficient production of energy or demand side technologies for reducing loads in energy end-using sectors. A general overview of the energy efficiency goals of the roadmap is as provided below.
Supply side technologies for the more efficient production of energy. The first goal in the Roadmap involves the development of technology that increases waste heat utilization in the industry, including electricity production from low temperature sources based on the Organic Rankine Cycle. The second goal focuses on combined heat and power and poly-generation energy technology, which requires the development of Stirling engines, heat exchangers, and absorption chillers among other components. Skipping to the seventh goal in the Roadmap, it is dedicated to the development of electric motors that comply with the premium efficiency standard of IE3 in Turkish and international electric motor regulation. There are a total of 3 goals that represent supply side technologies.
Demand side technologies for reducing loads in energy end-using sectors. The third goal of the Roadmap targets the development of highly efficient light emitting diode (LED) lighting armatures, which is seen to be a leading technology in the next 10 years. The fourth goal aims for the development of new insulation materials, such as vacuum tubes and thermo-chromatic materials. The fifth goal focuses on the design of advanced building information monitoring software that can control building-integrated renewable energy production and utilize phase-change energy storage as needed. The sixth goal involves the development of sensors to increase energy efficiency in buildings and the industry, including occupancy sensors. There are 4 goals for demand side technology.

Comparison of selected technology to trend analysis
As a means to compare the present trends of the selected technologies, a frequency analysis of articles in SCOPUS and patents in PatentScope has been conducted. According to the results as given in Table 1, it appears that all except four technologies have increasing trends in scientific articles and patents within the last five years. From a purely forecast point of view, this indicates that almost all of the technologies that have been selected as the outputs of the Delfi survey and focal group evaluations presently have upwards trends. On the other hand, the foresight nature of the Delfi statements depends on the technical specifications that have been assigned to these technologies that are to be developed. Therefore, it is not possible to base such a judgement on article or patent trends alone. However, such analysis when taken in combination with updated project portfolio analyses will be helpful in progress monitoring.

CONCLUSION
The comparison of the technology roadmapping process that was followed in Turkey for the preparation of the Energy Efficiency Technology Roadmap encompassed four of the five main steps in the proposal for an advanced version of transition management (v3). This case study from Turkey can also shed light on the discussions of an "integrated roadmap" under the Strategic Energy Technology (SET) Plan in Europe.
According to discussions of the SET Plan Steering Group, Horizon 2020 will emphasize an "integrated roadmap" that integrates skills, sectors and disciplines while grounding expected technological progress in sectoral dynamics [49]. The advanced version of transition management as introduced in this paper and demonstrated by the pilot case study of the Energy Efficiency Technology Roadmap in Turkey could satisfy aspects of an integrated roadmap in many other countries. In addition, given that the "Communication on Energy Technologies and Innovation" [50] indicates the topic of energy efficiency as one of the topics that will receive emphasis in the integrated roadmaps, it is envisioned that the case study will also gain importance in this respect.
The advanced version of transition management that unifies the concepts of "motors of change" and other elements of innovation system policy has the potential to provide a model for governance towards reaching a more sustainable energy future. This is unique among other technology roadmap cases, including those in the energy sector. There is reason to foresee that innovation system policy will have a key role in this more holistic framework for managing the transitions of complex systems that will determine our energy future. After all, we are in a moment in time when dynamics of innovation must be quickly shifting towards sustainability.