Energy end-use data collection methodologies and the emerging role of digital technologies
New and digital technologies have been unlocking opportunities to collect, manage, and analyze large amounts of data in a relatively cost-effective way. Still, given current challenges, it is prudent that their use for energy statistics is complementary to traditional methods until issues like data governance, confidentiality, or data representativeness are more widely addressed.
This aims at exploring the role of new and digital technologies for energy end-use data collection. It reviews applications, strengths, and weaknesses of the major existing technologies, classifying them into three broader categories depending on their purpose: data collection, data management, and data analysis.
The analysis is a starting point for energy statisticians and energy efficiency experts across countries in order to guide the design, and/or advise on the implementation of new technologies for data collection based on the case studies reviewed and on the analysis performed.
Data are the key to track policies’ effectiveness and to monitor trends over time, and energy data are no exception. In particular, disaggregated energy demand-side data collection has been a challenge in many countries worldwide, although the role of the demand-side of energy systems, notably energy efficiency, is widely acknowledged for delivering energy savings and avoiding emissions, and hence contributing to curb climate change.
In order to appropriately track energy efficiency progress, disentangle the different drivers of energy demand (such as activity, structure, and efficiency), and develop appropriate and detailed energy efficiency indicators, it is indispensable to have sub-sectoral or end-user data together with activity data with similar boundaries.
Increasingly, governments and organizations acknowledge the importance of and are committed to developing energy efficiency indicators across sectors, depending on national priorities, and collecting the relevant data. Traditionally, four main methodologies are widely applied for end‑use data collection: administrative sources, surveys, metering, and modeling. These are often used on a complimentary basis. Each has its own strengths and weaknesses, which are discussed in more detail later in this paper.
In addition to traditional methodologies, new and digital technologies represent an unprecedented opportunity for energy demand-side data collection to fill some of the most challenging data gaps as of now. Overall, new technologies can be categorized into three main types depending on their main purpose: data collection, data management, and data analysis. Increasing volumes of data collected in almost real-time, broad connectivity, and advanced data analytics could support end-user data collection and availability if properly streamlined and structured.
The inherent challenges and difficulties in the use of new technologies have been widely pinpointed in the literature. Huge amounts of data require proper management (including standardizing the data collected), ensuring data privacy, raising social acceptance, and allocating proper resources.
The bulk of the material for the section on traditional end‑use collection methodologies derives from the IEA’s Energy Efficiency Indicators: Fundamentals on Statistics, which has been expanded in the IEA Country Practices Database.
The digitalization section results of research work, alongside discussions from the 2019 workshop of the G20 end‑use data and efficiency metrics initiative – Uncovering the role of digitalization for energy efficiency indicators, and a survey conducted by the IEA on the role of digitalization for end‑use data collection.