The Urban Sustainability Assessment Framework (USAF) is designed as part of the Sustainable Cities Integrated Approach Pilot (SCIAP) project, which GEF-6 funds for Indian cities. Currently, it is being piloted in five cities in India [5]. It is envisaged to be a diagnostic and decision-support tool. UNIDO and UN-Habitat, in partnership with the Ministry of Housing and Urban Affairs (MoHUA) of the Government of India and state governments of pilot cities. To ensure integration among planning efforts and processes to evaluate their enforcement, the USAF brings together 12 sectors: governance and data management, finance and economy, housing and property, water, sanitation, waste management, clean energy, resilience, environment, public space-safety-urban form, transport, and social infrastructure. This framework has 131 indicators, with 235 data points across 12 sectors. One of the unique features of this framework is the spatialization of indicators [5]. The USAF is based on four existing national and six international frameworks and indicators recommended by experts. The NITI Aayog (Sustainable Development Index) framework was the second framework examined. The world has targeted reaching the Sustainable Development Goals (SDGs) by 2030 and is now a critical moment in the journey towards it. Various countries around the globe are investigating the progress made to date under the framework of SDGs and are trying to implement actions that will expedite the success of reaching the 2030 target. Acknowledging the difficulty in implementing 17 goals, 169 targets, and 306 national indicators given by the United Nations, the National Institution for Transforming India (NITI) Aayog, a national-level public policy think tank and nodal institution for implementing the SDGs, adopted the UN SDG Index for India. Since 2018, the SDG India Index has become the primary framework for measuring progress and providing inputs for data-enabling action towards achieving SDG targets [14].

In 2018, the first edition of the index was launched with 62 indicators under 39 targets across 13 SDGs. Some goals, such as 12, 13, 14, and 17, were not considered because of a lack of state-wise data under the specific indicators. In the second edition of the index, launched in 2019, all goals were covered along with 54 targets and 100 indicators. Of these, 68 were aligned with the National Indicator Framework (NIF) [14].

The last index for this study is the Ease of Living Index (EoLI), published annually under the India Smart Cities Mission, which has sustainability as one of its pillars. The ease of Living Index (EoLI) is developed to evaluate the quality of life, and the impact of various initiatives for urban development EoLI is used as the assessment tool. Based on a city's quality of life, economic ability, sustainability, and resilience, this study provides a holistic analysis of 114 cities across India. One part of the assessment also includes citizens' views on civic services the Urban Local Bodies (ULBs) provide. The EoLI is organized around four pillars, namely the Quality of Life, Sustainability, Economic Ability, and Citizens Perception Survey, to evaluate the well-being of Indian cities across 111 cities. Based on these four pillars, the 49 indicators were categorized into 14 categories [15] [16].

As one can see from Table 1 categorization of the USAF and EOL index is based more on functional aspects, with some aspects of resilience and clean energy covered. As the name suggests, SDG goals are categorized as more aspirational. However, most indicators under these aspects are outcome-oriented and make minimal attempts to capture the processes. Moreover, the categorization and focus on outputs suggest a weak sustainability approach that operates from a belief that clean energy interventions can justify continuous resource extraction. None of the frameworks focus on carrying capacities or the rate of resource extraction like water.

As per the USAF document, the framework was developed in various iteration phases. The first step was to undergo a detailed analysis of existing national and international frameworks, which were significant in examining the sustainability of cities. To create an easy-to-use and dynamic framework, the 131 indicators are categorized into 3 levels --- primary, secondary, and tertiary. Subsequently, each level is divided into sub-levels, allowing the cities to penetrate deeper into a sector of their interest or need. To emphasize the spatial aspect of these indicators, they are further categorized into spatial and non-spatial. Some hands are scored quantitatively ranging from very low to excellent performances. Therefore, a seven-point scale is used to measure the city's performance. Hence, a city should aspire for the "excellent" category being the highest of the levels. This scoring scale has been developed according to the national standards and global benchmarks wherever possible Figure 1.

Figure 1:

Sectors of the Urban Sector Assessment Framework and Scoring Scale[**]

USAF is still at the drafting stage and has been designed to cover as many cities as possible and can be applied to any Class I town, i.e., a population of more than 1 lakh. The target audience of the framework is municipal corporations, planning development authorities, Civil Society Organisations (CSOs), and private sector players.

Mixing of outcome and input indicators, lack of focus on process indicators and its continuity are some of the issues that the framework has to address. For example, the framework for functional sectors like housing, social infrastructure, etc., mainly uses outcome indicators like the availability of schools within a 500m distance. However, for aspects like governance, it uses input indicators like expenditure. The framework has very little focus on processes like decision-making, leadership etc. As it is meant to be a diagnostic tool, its main objective is to provide spatially organized information to decision-makers. However, if it remains only a onetime exercise, its advantage is minimal as it will be unable to capture change.

In addition to the above, the framework promotes weak sustainability discourse rather than a fundamental shift in approach. It therefore misses out on some key indicators like accessibility and carrying capacity. As part of the studio exercise (2023) on Planning for Sustainable Communities, students reviewed the applicability of the framework for a small hill town Chiliyanaula in Uttarakhand, India. Their work revealed a breach of carrying capacity for a small town of less than 5,000 population given their resources and current usage. Similarly, for schools currently the framework measures it through proximal distance criteria. Proximity criteria give a sense of the availability of a facility, its relevance in the Indian context is highly limited as students don't need to be admitted to neighbourhood schools. It has to be supplemented with accessibility criteria. In Chiliyanaula while there was a school available nearby it was a very expensive school and not at all accessible to the local population. Therefore, indicators focussing primarily on the proximal distance of facilities like schools in the Indian context fail to capture the accessibility of such facilities for diverse groups, resulting in misleading assessments. The indicators in their present form thus reflect a bias towards quantifiable indicators.

The SDG Index by NITI Aayog offers insights into social, economic, and environmental status and the States/UTs' journey towards achieving the SDGs.

Out of the 115 indicators in the current index, 75 are expected from the previous index, i.e., Index 2.0. The values of 57 out of these 75 have been updated. Furthermore, 76 out of 115 indicators are entirely aligned with NIF, 31 are derived from NIF, and 8 are constructed in engagement with the line ministries. For assigning any weightage in the index estimation, the missing cells were considered "Null" due to the unavailability of data under those indicators for all States / UTs Figure 2.

Figure 2:

Methodology adopted by the SDG Index[††]

The indicator development process and its operationalization were through the involvement of the Ministry of Statistics and Programme Implementation, 38 central ministries and 36 state governments and Union Territories, and experts Figure 3. There have been attempts by the government to involve the next tier and the local bodies. As part of the SDG Index, all state governments must set these targets for 2030.

Figure 3.

Institutional Structures at Various Levels in India for SDGs[‡‡]

Thus, while there is an awareness of the idea of sustainability through these indicators unless the process of its operationalization does not include a discussion on the form and direction of such development, the real estate interests will continue to prevail in cities contrary to the purpose for which indicators are being formulated.

The Ease of Living Index (EoLI) with four pillars, i.e., quality of life, sustainability, economic ability, and citizens' perception survey, make up the EoLI evaluating the well-being of the Indian cities across 111 cities. Under these four pillars, 49 indicators were categorized into 14 categories. The different aspects contributing to a decent urban life in a city fall under the first pillar of "Quality of Life," which holds a weightage of 35% in the final index score. The level of economic development and inequalities an individual encounter in a city falls under the "Economic Ability" pillar and holds 15% weightage in the index score. "Sustainability" is the third pillar and comprises the availability of green spaces, green buildings, energy consumption, air quality, water quality, and the city's capability to withstand natural disasters, and a weightage of 20%. The fourth pillar, i.e., the "Citizen Perception Survey," was conducted to strengthen the index further with 30% weightage.

Figure 4.

Four Pillars of EoLI[§§]

A thorough analysis was done of the cities with a population of greater than 1 million as per the projected population till 2019, and all cities covered under the India Smart Cities Mission (regardless of their population size). Therefore, 111 cities were selected for evaluation in the Ease of Living Index. The data collected for the 49 indicators across the Index had been obtained in various units like Percentage, Ratio, and Binary Marking.

The India Smart Cities Mission has undertaken multiple performance evaluation initiatives like EoLI, MPI, Climate Smart Cities Assessment Framework (CSCAF), and Data Maturity Assessment Framework (DMAF). The primary purpose behind these indices is to assess the cities across varied sectors that impact the nature of growth in the cities. Previously data has been collected through various frameworks at different times and cycles, thus having issues like overlapping, duplicity, and more time and effort being expended in the exercise. Therefore, through the Urban Outcomes Framework, data across 14 sectors have been streamlined so that there is an increased focus on data collection, and disaggregated data can be analysed and used for ranking by domain experts. This also provides the opportunity for creating an ecosystem to create new frameworks based on open data. This exercise aims to shift focus from indices to data with a comprehensive list of indicators. The cities will regularly update the datasets, and data points will be accessible in the public domain for the consumption of researchers, academics, and public audits [17].

The process of formulation of indicators included first set of indicators being prepared by the mission unit of the Ministry of Housing and Urban Affairs at the national level. Four one-day national level consultations were organised where city data officers were invited to give their inputs. Based on these consultations, indicators are finalised, and city level data collected. Feedback mechanism is only through consultation.

The biggest challenge in implementing responsive planning decisions and evaluating their output is the inadequate, reliable, verified, and up-to-date data at a regional level. ULBs are the biggest custodians of granular data and are responsible for verifying and submitting data as per the indicators. In the current version of the SDG India Index 3 (2020–21), out of the 115 indicators, 26 are from 2020–21, 31 from 2019–20, 34 from 2018–19, 6 from 2017–18, 9 from 2016–17, 7 from 2015–16, and 2 from 2011–12. Since old data from 2011–12 to 2018–19 are used, the composite index ignores the state's current performance_level in those domains. The conditions may have progressed very differently concerning these indicators. For example, the latest data regarding the population below the poverty line is as old as 2011. In addition to unavailability of real time contextual data, this has to be organised as meaningful information for any effective participation to happen. In the absence of meaningful information combined with absence of consultation at the local level, ownership of SDGs at the local level remains disjointed.

Consultation workshops were conducted with sector experts in the USAF framework formulation process. These domain experts gave inputs to the indicators as per existing national and international guidelines. In case of sustainability indicators by NITI Aayog as well as EOL indicators too, there has been limited consultation with the State government and municipal officials. The states are expected to improve their performance in these areas, and there is no coordination between all the stakeholders on the indicators, their data flow, and aggregation formula among all stakeholders before such an exercise is undertaken. Consulting the state and ULB officials would help evaluate their performance and reach the target level. Otherwise, it might pose challenges for methodical validation on state-level data flows. A state may disagree with the indicator value for politically sensitive barriers. It is, therefore, essential to develop systematic data flow and validation mechanisms at the state level and to build the capacity of states for proper reporting and monitoring of SDGs.

Moreover, while there is some form of parliamentary oversight, the consultations rarely focus on values underpinning the indicators. Most of the discussion focusses on feasibility of indicators. The consultation process thus has to deliberately include discussion around values and value conflicts. Absence of indicators related to carrying capacities, distribution of land resource and accessibility aspects reflect a weak sustainability approach. The documents do not reflect any value dilemmas and the approach thus the process of consultation remains very instrumental in nature.

Design of indicators reflect relationship between the outcome and its measure. It would include a process of selection and inclusion, decisions around the level at which this should be measured and choice of measurement scale. As mentioned earlier, expert consultations around selection and measurement is part of the process. However, the option of localising these indicators is minimal.

Some of the indicators in the USAF frameworks are on a seven-point scale ranging from very poor to excellent performance. This scale is qualitative, considering the perception of an individual based on the data inputs submitted by the cities. This means as per one individual; the rating might be medium; however, for another individual, it might be upper-medium performance.

Similarly, in case of NITI aayog, 17 SDGs must be measured in combinations of different dimensions, comprising several socio-economic outcomes like poverty, health, education, inequality, environment, quality of life, and infrastructure development. This means NITI Aayog uses a composite index score to represent the progress made by the States in India under the SDGs. However, using a composite index has been questioned multiple times for combining various dimensions in one number. In the SDG Index, each indicator is normalized, after which the arithmetic mean of the indicator values is used to arrive at the composite score. Hence this means equal weight for every hand, and as a result, it is a compromise between the loss of information and being able to communicate transparently. Therefore, a partial-compensatory composite index like the geometric mean is more suitable for the SDG India Index, provided there will be no indicator with a "null" score value.

Introduction of new indicators and changes in measures of indicators makes it difficult to undertake temporal comparison. A state that ranked better in a particular goal in the previous edition of the index could become a worse performer as new indicators are introduced. Such abrupt and unexpected changes in the state's rank may create misinterpretation and affect the integrity of the SDG monitoring process. Thus, it is crucial to consolidate the list of specific indicators through the consensus of all the state officials to measure particular goals that may be used to assess performance.

A huge disincentive for states is due to the capping of the normalized score when the performance is above the target. Capping the score to the target value, that is 100, when the state achieves a score beyond the mark, e.g., 120, makes the states demoralized, which can improve their performance beyond the target.

Subjectivity in measurement as well as combining complex set of concepts in single indicator and city rank is convenient and simple. However, this encourages more focus on improving the rank (which is quick and amenable to manipulation) rather than changing processes (which is slow), hampering its efficacy in effective policy interventions. The indicators related to process are not given as much weightage and importance as the outcome indicators.

As mentioned earlier, national level SDGs get translated by each state with targets for the year 2030. Most of the states in India have set these targets for 2030. However, institutional capacities, top-down target setting and interdepartmental coordination makes their implementation an uphill task. Here we would briefly like to discuss an example of a small hill state of Himachal Pradesh in India. Himachal Pradesh, a hill state in India, has set SDG 2030 targets at state level [18]. The state has assigned responsibility to various departments and organisations for implementation of targets. Most organisations have a staff crunch to be able to effectively work on these targets.

One of the indicators requires urban plans to be prepared for all towns in the state of Himachal Pradesh, India. This is the responsibility of the town planning department. The department currently has a director of town planning who is a bureaucrat along with a district town planner and assistant planner. None of them have a degree in planning. The current development plan for the city was prepared by private consultants from Ahmedabad, Gujarat [19].

Being statutory in nature, development plans can be a strong instrument for implementing sustainable goals. However, environmental protection and sustainable development are not the stated goals of the plan. Similarly, the plan, while developing population scenarios for population projection, does not consider sustainability as a criterion [18].

One reason for the limited focus on the sustainable development approach can be attributed to the process through which such indicators are formulated [10]. In a top-down approach, the list of indicators is handed over to the staff in different departments. However, its purpose and significance gets rarely discussed. Discussions on the allocation of both financial and human resources are usually absent at the local level. Data is hurriedly collated without focusing on institutional learning or systemic change. Moreover, real estate private interests prevail much more than sustainability goals in plan formulation. With limited capacities and larger private interests, SDG indicators become another task and fulfilment of a formality rather than a substantive shift at local level [20].

The Shimla Development Plan, a statutory plan, thus became quite controversial for not adhering to sustainable development principles. The National Green Tribunal (apex body for the effective resolution of legal cases on environmental protection) placed a stay on the development plan project because it promoted unsustainable development. In one of its earlier orders in the matter of CWPIL NO 13/2021, Kusum Bali vs. State of HP (Himachal Pradesh) and others dated 13.01.2023, it stated:

"The State, as a trustee, has a legal duty to protect the natural resources and the environment and prevent its degradation under the 'Public Trust Doctrine.' Moreover, there has been non-observance of principles of 'sustainable development' as well as 'precautionary principle" by the State, which envisages that if there is the risk of severe damage to humans and the environment, absence of incontrovertible conclusive or definite scientific proof would not be a reason for inaction."

As can be seen from the discussion above, existing frameworks focus more on the outcome and less on the process. Required database systems in towns and cities are currently non-existent. Thus a lot of time is spent generating data for different frameworks in different formats. Many aspects, like housing and water supply, do not capture existing capacities to supply or provide housing there, leading to incorrect diagnoses. Lack of real time data, temptation to improve rank and low ownership of SDGs at local level affects the monitoring and feedback process. For example, while many of the towns report themselves as open defaecation free, lack of basic sanitation still eludes most of the towns.