Energy transition pathways amongst low-income urban households: A mixed method clustering approach

Abstract

Studies on clean energy transition amongst low-income urban households in the Global South use an array of qualitative and quantitative methods. However, attempts to combine qualitative and quantitative methods are rare and there are a lack of systematic approaches to this. This paper demonstrates a two stage approach using clustering methods to analyse a mixed dataset containing quantitative household survey data and qualitative interview data. By clustering the quantitative and qualitative data separately, latent groups with common characteristics and narratives arising from each of the two analyses are identified. A second stage of clustering identifies links between these qualitative and quantitative clusters and enables inference of energy transition pathways followed by low-income urban households defined by both quantitative characteristics and qualitative narratives. This approach can support interdisciplinary collaboration in energy research, providing a systematic approach to comparing and identifying links between quantitative and qualitative findings.•A mixed dataset comprising of quantitative survey data and qualitative interview data on low-income household energy use is analysed using hierarchical clustering to detect communities within each dataset.•Interviewees are matched to quantitative survey clusters and a second stage of clustering is performed using cluster membership as variables.•Second stage clusters identify common pairs of survey and interview clusters which define energy transition pathways based on socio-economic characteristics, energy use patterns, and narratives for decision making and practices.

Keywords: Clustering; Data science; Energy access; Mixed methods.

Graphical abstract

Fig. 1

Schematic overview of mixed method cluster analysis. The analysis starts with a first stage clustering of both the quantitative survey data and the qualitative interview transcripts. Hierarchical clustering methods are used in this first stage, with the interviews coded for the qualitative clustering. The interview respondents are then matched to a corresponding survey cluster based on common socio-economic variables and a second stage of clustering is performed to identify distinct combinations of qualitative and quantitative clusters that characterise different energy transition pathways.

Fig. 2

Schematic of mixed data structure. Two distinct datasets are required: one contains qualitative household level survey data with a mix of energy and socio-economic variables, the other dataset consists of a table of codings applied to the transcripts of interviewed households.

Fig. 3

Example of use of Silhouette Width and Slope Statistic to determine optimal number of clusters for a sample of households in Bangalore. Note how while there are maxima on the silhouette width plot at 5 and 16 the slope statistic indicates 5 as preferable.

Fig. 4

Schematic of interview coding clustering process. Coded interview table is transformed into an adjacency matric by calculating correlation between respondents. This adjacency matrix is then clustered to assign interviewees to respective clusters.

Fig. 5

Network graph of communities detected amongst interviewees based on interview coding correlation. These groups form the qualitative interview clusters.

Fig. 6

Schematic of interviewee survey cluster matching based on socio-economic variables.

Fig. 7

Schematic of second stage clustering and pathway characterisation. K-means clustering of the interview households using their qualitative and quantitative survey clusters as variables identifies pathway groups with unique pairs of.

Fig. 8

Graphic showing pathway cluster membership, associated socio-economic characteristics, and key qualitative interview data codes for each pathway in the example of low-income households in Bangalore.