India’s ESMI

Across the developing world, roughly 1.2 billion people3 do not have access to electricity. Of this number, at least 30 percent live in India. In addition, at least 247 million people in India experience irregular access to electricity, with many receiving only around four hours a day. The government of India, under Prime Minister Narendra Modi, has committed to establishing universal access across the country. However, India’s electricity problem is not just about insufficient coverage; it is also about poor power quality, especially in the form of voltage fluctuations.

Poor power quality impacts all segments of consumers; it can damage equipment and cause various other problems, including data loss and other forms of loss or inefficiencies for businesses and other entities. Improving power quality is, however, challenging, and requires real-time access to data. In 2007, The Prayas Energy Group (PEG), an Indian NGO, launched the Electricity Supply Monitoring Initiative (ESMI) to complement existing data sources and collect real-time power quality information by installing Electricity Supply Monitors (ESMs) in various locations. ESMI now works in 200 locations4 in 18 Indian states. The initiative has made power supply monitoring data available for different users across the country, in the process increasing awareness of the state of electricity supply, helping to advocate for better service provision, and influencing policy at both the state and country level. Despite certain limitations—and the daunting scale of the problem—ESMI represents an important example of how civil society can address public problems, and fill gaps arising from government data failures, when NGOs take an open approach to data collection and use.

Prime Minister Narendra Modi, elected in 2015, included universal access to electricity as one of his top priorities. The government has devoted considerable resources to a rural electrification program7 that aims to electrify 121,225 un-electrified villages, improve power supply to 592,979 partially electrified villages, and provide free electricity to all rural households. As of June 30, 2016, it is estimated that between 50 and 80 percent of these targets have been achieved.

Nonetheless, the quality of electricity service provision remains poor, as emblematized by India’s infamous 2012 blackout8, in which 680 million people were affected. The root causes of the problem are manifold. Distribution on India’s antiquated and poorly connected grid is one of the main issues; by some estimates, up to 30 percent of the nation’s electricity is lost in grid inefficiencies. But the problem isn’t only about distribution. The nation simply doesn’t generate enough power. According to a 2015 report from the Institute for Policy Research9, an independent policy research institute, the nation’s power deficit stands at 3.6 percent. Even this figure probably underestimates the problem, given that energy statistics calculate demand only on the basis of existing connected consumers, meaning that such power deficit figures fail to take into account the millions who do not have power at all.

One of the biggest, though perhaps least acknowledged, problems concerns the quality of India’s power supply. Power Quality10 encompasses voltage variations (sags and swells), voltage reductions, power interruptions, voltage surges and harmonic distortions in the supply. Simply put, poor power quality refers to interruptions in power supply (which might last from a few minutes to a few days) or dangerous spikes in supply that could damage household electronic devices.

Such quality problems are widespread in India and they have major implications for domestic as well as industrial consumers. While it is difficult to quantify the consequences, one study conducted in 200911 suggested that businesses had to invest 15.5 billion USD in back-up power generation facilities to avoid the various adverse impacts12 of poor power quality. These adverse effects include frozen computer screens, data loss, flickering lights and equipment damage, among other issues.

Addressing power quality issues represents a considerable challenge—in some ways, more so than addressing the problem of non-existent supply. To address the issue, real-time data is required, and such data is hard to come by. Regulators, electricity producers and consumers need as much information as possible on when electricity is likely to be cut or spike, and on what factors typically trigger voltage fluctuation. Data that can help to predict power fluctuations and provide forewarnings on voltage dips or spikes are also extremely important.

To make progress toward meaningfully addressing the many current problems in the Indian energy sector, a far more sophisticated data setup is needed. According to a recent study13, there are different regulations on power quality in India, issued by the Central Electricity Authority (CEA) and also by the State Electricity Regulatory Commissions. There exist several shortcomings and variations in these regulations, notably when it comes to enforcement, but also in the way data is handled or treated. For example, voltage variation limits14 for some states is 10 percent but for others it is 12.5 percent. Across states, there exists no reliable voltage monitoring program to provide power quality data. The only way to report low voltages or power outages is to call the local power company and lodge a complaint—a time consuming, manual process, which in any case is unlikely to have any meaningful impact.

Such shortcomings affect policymakers and those who seek to improve power quality. Consumers are also directly affected by a lack of data, for example in lodging complaints against distribution companies regarding problems in power supply. Without data, they have no evidence to back their claims. While there do exist clear standards regarding acceptable variations in supply parameters, defined15 by the Institute of Electrical and Electronic Engineers (IEEE), citizens or even regulators often have no reliable data to show that those standards are not being adhered to.

Open Data in India

The Government of India approved the National Data Sharing and Accessibility Policy (NDSAP) in early 2012. This policy can be considered the first enabling regulation regarding the proactive disclosure of government data and extends the mandate of the Right to Information Act (RTIA). The act established policies and procedures in the publication of government datasets from different agencies in the central government through a single national portal.

The national open data platform16 was launched in the same year. Since then, the portal has become the main repository of government data sets, covering 102 departments and involving at least 111 chief information officers. To date, the portal houses 44,174 resources in 4,043 catalogs covering essential sectors, such as health, environment, education, agriculture, commerce, mining, legislation, labor, power, energy, and tourism, among others. However, the 2015 Open Data Barometer17 has shown that despite the increasing number of data sets, there is little evidence of impact on government’s accountability, effectiveness, and efficiency and in environmental stability or social inclusion. The Open Data Index places India at 55 percent open and ranked the country at number 15 in its 2015 edition.18

The power and energy sector database in the national portal consists of at least 137 data catalogs, 47 of which are dedicated to electricity and power. The others include information on areas like the functioning of thermal power stations, efforts to expand the use of renewable energy and the supply of and demand for natural gas. None of the datasets includes information on power quality; they are largely focused on power generation. The data catalogs are also difficult to navigate, as several of the datasets are structured by state and are not linked in any way that would permit a national or otherwise aggregate analysis.

Initially, there was some skepticism regarding the viability of a technology-based solution for monitoring electricity supply. Concerns included the possibility of mobile network failures (the ESMs transmitted information via cellular networks), the difficulty of finding field volunteers to host the monitoring devices, and the challenges of developing a low-cost monitoring module. In addition, some questions were posed about the need for such a system in a rapidly developing environment of intelligent hardware (e.g., smart meters) and software tools (e.g., Supervisory Control and Data Acquisition, or SCADA, systems, which provide industrial monitoring and control mechanisms for complicated processes like energy distribution).

Over time, however, it became apparent to all stakeholders that there was indeed a pressing need for a transparent and automated data collection system. As of May 2016, ESMI covered 200 locations in 18 states, with 1.5 million hours of power quality26 monitoring data available.27 It has also become apparent that the project can play a powerful developmental role, in particular by helping to address certain inequalities in power distribution and consumption. As Shweta Kulkarni, a Research Associate at Prayas and one of those chairing the ESM project, put it:

Most energy sector policies focus on financial viability and economic growth while good governance and equity remain neglected. This often leads to poor supply quality, especially in marginalized neighborhoods. One of the important avenues to change this situation is to improve the transparency and accountability in the sector by creating a publicly-accessible database on supply quality information to show the variations in supply quality between urban and rural areas, different states, distribution companies and areas of economic importance.28

Funding

ESMI was one of six finalists in the 2013 Google Social Impact Challenge awards. Each finalist was awarded 15 million (roughly $225,000) in seed funding. With the funding, Prayas was able to install power quality monitors in 60 locations across eight states, including in at least five megacities.29 That initial coverage has expanded rapidly in the years following.

Demand and Supply of Data type(s) and sources

ESMI has made it clear that good power quality management is impossible without real-time data. This is true not only for consumers, who suffer the most direct impact of poor power quality, but also for electricity producers and distributors, and even for regulators. For producers and distributors, real-time power quality data is important in identifying problems in production, transmission and distribution. Such data helps them identify and potentially address shortcomings in their own processes and management. In addition, regulators can use data to hold private companies to account, potentially assessing fines or other penalties for under-performers; overall, data helps regulators ensure the quality of electricity distribution across the country.

The institutional arrangements, in this case, are complex. Demanding accountability regarding power quality in many ways creates an endless cycle of buck-passing from one stakeholder to the other. While grievance and redress mechanisms are enshrined in India’s Electricity Act30, through a legalistic and layered procedural mechanism, use and impact of this process is hardly visible, and normally ends in frustration and subsequently, consumer apathy. A few organizations, however, focus time and resources into research and advocacy on power issues, among them The Energy and Resources Institute31, monitoring groups like Andra Pradesh’s People’s Monitoring Group on Electricity Regulation, and Prayas.

It is important to note that some useful open energy data did exist before the advent of the ESMI project. For example, the Ministry of Power32 and the Central Electricity Authority33 (CEA) have released several data sets on the national open data portal (data.gov.in). These are available for free download. However, the bulk of this data relates to power generation, supply and demand, and tariff information. For example, the government publishes the number of electrified villages and other information pertaining to its monthly rural electrification mission on a monthly basis. As of late 2016, there is virtually no data available to gauge the quality of power—hence the vital importance of the data being generated by ESMI.

Open Data Use

All the data being used for this project are generated by the ESMs installed by Prayas. ESMs are “smart, connected energy meters” that can accurately measure voltage fluctuations as low as 90V and as high as 320V. The devices use very little power; they simply need to be plugged into a socket and immediately start measuring and transmitting voltage data over 2G/3G networks using an internal SIM card. Currently, three Indian companies supply Prayas with ESMs, each of which also offers a one-year maintenance contract: Syslabs Automation Pvt. Ltd., HelioKraft Technologies Pvt.Ltd, and Altizon Systems.

The data generated by the ESMs is made available for free at a website set up by Prayas34. The data is presented in three different forms: (1) minute-by-minute voltage information of all monitored areas; (2) reports that analyze voltage data for each location; and (3) more general analysis of the aggregated data that considers the voltage situation at a regional as well as the national levels. The analysis is often presented in the form of reports, which typically include information about the number and duration of power disruptions (including fluctuating voltages and frequency of supply). A sample report is shown in Figure 1.

Figure 1. ESMI Sample Analysis of Power Quality

Currently, data cannot be downloaded from the portal in bulk, and instead must be downloaded for a single location at a time. Users can view data for a given 31-day period on the website itself, but data for up to 100 days can be downloaded and viewed offline. Downloading of multiple locations or for longer periods is only possible through a user-request made directly to Prayas. Prayas assesses the request and can choose to grant a one-time username and password to access bulk data. Overall, the system requires very little technical knowledge, and the easy availability of data on a digital platform adds a new, automated level of transparency to the industry.

At the project’s inception, Prayas envisioned that several different types of users would be able to use the data and make a positive impact on the power situation in India. Consumer rights groups would be able to implement evidence-based advocacy on power quality. Government regulatory agencies at the country and state levels would be able to use the data to monitor performance of power generation, distribution, and transmission companies, and to identify new policy and legal interventions to improve the power situation. Prayas also expected researchers to use the data to find new mechanisms to improve power quality across the different states, as well as to identify alternative sources and channels for better power service delivery.

While direct evidence of impact is often hard to capture, it is clear that many of these envisioned uses have, to a greater or lesser extent, in fact been borne out. An internal assessment conducted by Shweta Kulkarni, the person within Prayas responsible for the program, claims that ESMI has had an impact in bolstering, consumer satisfaction, improving power quality improvement, and enabling evidence-based advocacy.

However, perhaps no surprise due to the newness of the project and the many stakeholders needed to make meaningful progress action, at this stage we do not yet have documented cases that show the power quality has actually improved as a result of the advocacy mentioned above using ESMI data (outcome-level results) or improved satisfaction of consumers because of improved service (impact level results).

As Dr. Pendse puts it:

“The question is, in the 200 houses that it (ESMs) has been placed in, has the power quality improved? And if it hasn’t then at least has someone been made accountable using this evidence? The whole idea is that the use of data should lead to improvement. Otherwise, it isn’t working.”

Nonetheless, in our assessment of the evidence, through desk research and interviews with key stakeholders, we do identify the following three main areas of impact.

Awareness Raising

ESMI has been used by various individuals and organizations to raise awareness on the serious power quality problem faced by India. One of the most important constituencies for the data are journalists who write on power supply conditions.35 Several of these journalists have raised the profile of issues related to power quality in India, which are often eclipsed in public discussions by the topic of electricity access.

In addition, Prayas was able to present its project and findings to the CERC’s meeting of regulators, also raising their awareness of power quality issues.36 As a result, some state-level energy regulatory commissions volunteered to use ESMI to monitor the performance levels of utility companies that fell under their jurisdiction. We discuss this issue further below.

Effectiveness of power quality advocacy

There is some clear evidence that data from ESMI has been effective in enhancing the efforts of those advocating for greater power quality. For example, in the Akola Industrial Area, in the state of Maharashtra, data recorded by an ESMI device was presented by a consumer to the local officials of the distribution company. This data proved that the Akola Industrial Area had experienced unplanned interruptions and other power quality problems which caused loss and damage to industrial products and equipment. The local distribution company officials were thus to acknowledge the problem, and to plot out some remedial actions.

The ESM reports available on watchyourpower.org, also helped identify a number of lapses in the system including non-adherence to protocols for load shedding37 and exposing loopholes in claims of zero power deficit by the government.38 There has been little evidence that these insights have led to any concrete actions, however. A common challenge across this series of case studies is a lack of institutional responsiveness to act upon insights generated through such data analyses.

Re-emphasizing power quality monitoring at the regulatory level

As mentioned, ESMI has helped to raise awareness of power quality problems. While journalists and advocacy groups have been among the most important target groups, ESMI and its data have also succeeded in raising awareness among regulators and more generally putting the issue of power quality back on the regulatory table.

ESMI has been presented at several regulatory meetings, including the Forum of Regulators, the Delhi Electricity Regulatory Commission, and the Joint Electricity Regulatory Commission—each of which plays a different but important role in the national and state-level regulatory framework. In addition, ESMI data have also been presented to private distribution companies and researchers at various forums and seminars. ESMI has also received support from the Joint Electricity Regulatory Commission to install ESMs in the Union Territory of Chandigarh and State of Goa.39 As of late 2016, Prayas is in dialogue with another regulatory commission and distribution utility which agrees to support deployment of ESMI in their areas of operation.

Risks

As evidenced by the example of ESMI and various other case studies included in this series, open data holds tremendous potential for positive transformation. But as we also see throughout this series, open data also poses certain risks. It is important to understand these risks in order to ensure that open data projects are implemented in as safe a manner and in a way that maximizes the potential upside and limits the downside.

The problem of power quality is multifaceted. While power quality data at the consumer level is useful to emphasize a consumer’s complaint, as Dr. Priya Jadhav, Associate Professor at the Indian Institute of Technology-Bombay points out, utility companies are often aware that the power they are supplying is low voltage. Data such as ESMI only re-emphasizes what is already known. But because of the poor financial situation that the distribution companies are in, they are in no position to fix the underlying issues that result in poor quality. While identifying the areas where poor quality is persistent is a useful piece of the puzzle, the real solution may lie in understanding why distribution companies suffer losses (electrical as well as financial) and rely repeatedly on government bail-out packages. To cut their losses, distribution companies often do not purchase power from the power generation companies, which explains why many places in power-surplus India suffer so many power outages.

So the risks of ESMI not succeeding lay at its very strength—that of providing power quality data. Given that there is already a sophisticated way of monitoring power quality and providing the results of this monitoring to the public, the expectation that this will be able to improve power quality supply may not necessarily happen as users will not use the data and advocate for power reform, or that generation, transmission, and distribution companies will not be made to act on the transgressions that the companies committed. Without power supply improvement, ESMI will just become, sadly, a resource useful for researchers and journalists with no real difference to power consumers, especially those at the base of the triangle.

The Prayas initiative has brought to the fore why power quality monitoring is important, and it has re-emphasized a problem that has long been ignored by power sector stakeholders. It is now clear that, properly scaled, ESMI has the potential to identify under-performing power companies and regulators, impose a level of accountability in India’s power sector, and, most importantly, contribute to better power quality.

This section considers some key lessons learned from the project—lessons that are potentially applicable to other open data projects, in other parts of the world. We split our analysis into a discussion of Enablers (positive lessons) and Barriers (negative lessons); both are equally important to better understanding the impact and implications of a project like ESMI.

Enablers

High Level Vision

The India Smart Grid Task Force,40 which was set up in 2011, released a “Vision and Roadmap” document in 2013,41 recognizing that improving reliability and quality of power to consumers was one of the key drivers of the smart grid.42 Real-time monitoring, automated outage management, and faster restoration are some of the key targets of the Smart Grids in order to improve overall power supply quality across the country. While this was not achieved, ESMI took hold at a time when political and public will to leverage technology to improve power supply was at a peak. Drivers of the effort at Prayas point to this growing priority among the public and institutional stakeholders helped to clear the way for the ESMI initiative. This background also points to the ways in which open data can be used to evaluate performance, but a set of indicators or expressed expectations are important for guiding efforts and evaluating success.

Collaboration between Civil Society and the Private Sector

The success of ESMI is in large part attributable to the vision, commitment and hard work of Prayas. It was Prayas’ prior work in the field of electricity that led the organization to look for new ways of monitoring power supply quality and enforcing accountability. The existence of a civil society champion such as this one, with experience and contacts in the field, cannot be underestimated.

Prayas’ vision was critical. But its ability to scale ESMI to more than 200 ESM locations was boosted by an external grant from Google. In addition, the existence of private companies able to manufacture the ESMs at low cost, and with a commitment to service the monitors, also helped ensure the success of the project. Overall, it is clear that ESMI was the beneficiary of an existing (if under-exploited) ecosystem of both private sector companies and civil society organizations. This inter-sectoral collaboration offers a powerful model for other similar projects around the world.

Barriers

ESMI has seen some success, but it has not managed to really turn the tide on power quality issues in India. Its lack of wider impact also offers lessons, specifically:

Lack of use of ESMI data from key power stakeholders

Despite enthusiasm among existing users, ESMI’s data has not really found a critical mass of users that could help scale the project and hold power sector stakeholders accountable. In particular, while journalists and advocacy groups have shown interest in the data, there has not been equal interest shown by consumers—citizens or corporates. This can be attributed to several factors, like the need for more awareness building efforts. The apathy may also have something do with a general sense of hopelessness and disempowerment among consumers; given the scale and long-standing nature of India’s power problems, consumers may not hold out much hope for a solution such as ESMI’s. Addressing such shortcomings may require greater coordination with intermediaries to spread awareness and confidence, as well as a concerted effort to introduce consumers to the very real potential of open data—both within the electricity sector in particular, but also more generally.

Lukewarm reception from state regulators

While Prayas has been relatively successful in presenting ESMI to power regulators, it is not clear that the willingness of regulators to listen and learn about the project is accompanied by the actual will to use it. Indeed, to date, only one state regulatory agency has actually developed a concrete plan to implement ESMI. This lack of regulatory will (if not interest) suggests that the inherent value of a project may not be enough to get it adopted and that Prayas needs to find new forms of leverage on regulators—e.g., through enhanced public or political pressure.

Lack of capacity on GTD companies to solve power quality problems

As earlier stated, power quality is a problem with a myriad of causes. ESMI data is only able to systematize the provision of data on already-known symptoms of the problem. But the deeper causes—why power companies are not able to supply quality power and why regulators are not able to provide penalties as well as incentives—remain unaddressed, and are unlikely to be solved through this type of initiative, regardless of the level of uptake.

Sustainability

Prayas believes that the sustainability of ESMI will depend in large part on collaboration and partnerships. Initially, Prayas faced significant challenges in developing its technical solution for remote monitoring of supply quality, owing to lack of in-house expertise in hardware and software development. But it initiated a dialogue with other stakeholders and shared information on the type of technology that it wants to develop to the tech community who are also willing to contribute to and support their goal. In this case, India-wide implementation is only possible if state regulators become convinced of ESMI’s value. Thus, Prayas is very actively reaching out to regulators and presenting in the regular regulators’ forum so that more support in scaling up and adoption can be gathered.

Replicability

India’s ESMI has generated considerable global interest. In fact, Prayas is now in discussions with different stakeholders to pilot ESMI in other countries. Discussion with stakeholders from Kenya, Tajikistan and Tanzania are currently ongoing. In addition, the first overseas pilot implementation is already happening in Indonesia, with the Institute for Essential Service Reform based in Jakarta as local partner.

The main goal of ESMI is to make available reliable data on the quality of electricity supply. Consumers, civil society organizations, researchers, regulatory commissions and other concerned actors can use this data to increase the accountability of electric utilities. The actual performance of companies can be compared to the standards of performance prescribed by regulatory commissions. In addition, utilities’ capital expenditures on improvements in supply quality can be scrutinized to verify whether supply quality has actually improved as a result of the investments. The openness of the collected data also enables multiple stakeholders to verify the government’s many goals for rural electrification, notably whether villages receive their mandated six hours of daily supply. Overall, the data generated and opened by ESMI can create greater pressure on regulators and utilities to become more transparent, more accountable and more efficient. The project offers great social and developmental potential.

In order for ESMI to truly reach its potential, stakeholders need to know that the data exists and is accessible. This remains a challenge, and a work in progress. For all its initially impressive results, scaling up ESMI will require a concerted effort to raise awareness about the possibilities offered by open data among average citizens, corporate consumers, and government regulators. In addition, ESMI could benefit from greater political support and public will to ensure that regulators actually use its data and take advantage of the many benefits it does potentially offer. The opportunity—for India’s electricity sector, and more generally for its social and economic development is real; the coming years will tell whether Prayas and other stakeholders can seize that opportunity.