Skip to main content

Big data analytics is a new window of opportunity for the Indian water sector. 

The views and opinions expressed in this blog are entirely personal.

Data analytics and smart infrastructure are redefining utility performance and value addition across the globe. Though water is a reluctant late entrant, big data analytics, with its newly acquired speed and efficiency offers an amazing opportunity to unleash a new wave of reform; showing how to do more with less and how to improve the efficiency of existing investments. These advancements are all the more important in a resource scarce developing country. 

Challenges galore

Unfortunately, water is one of the key sectors in India that is sheltered and relatively untouched by reforms. Though demand responsive, decentralised cost recovery models have been tested since the late 1990s in the rural sector, but lost much needed steam. India is facing serious water challenges such as poor quality service delivery, a large proportion of un-served people, massive stock of languishing assets, and huge investment requirements. Sector interventions to address these deficiencies have been predominantly on the supply-side, but, pumping in more investments and creating infrastructure have not resulted in an improved service delivery.

As the Governments (both Centre and States) contribute over 95% of the sector financing and service delivery is heavily subsidised, water utilities have little or no incentives to perform.  The accountability mechanism is circuitous and in the absence of an independent regulator to discipline, benchmark and penalise inefficiencies; utilities are technically and financially unsustainable.

Though the Government of India (GoI) is committed to achieve the SDGs and going ahead with the setting up of 100 SMART cities, the road map for water is still unclear. Drinking water proposals under the SMART city projects are mostly refurbished old unfunded proposals without understanding the smart water building blocks integral to the water sensitive urban design. Despite cumulative investments of Rs. 4280 billion, India still has the largest number of rural people living without access to clean water – 63.4 million. The investment requirements by 2031 for urban water supply and sewerage alone are estimated at Rs. 5635.96 billion. Under the scenario of serious resource constraints, a major option is to push the utilities to a higher trajectory of efficiency and performance. Big data analytics now unfolds yet another window of opportunity for reform, provided the governments encourage the ‘’drivers’’ and steer the utilities to overcome the ‘’barriers’’. 

Big data analytics: a new window of opportunity

The water and wastewater industry is undergoing a transition from typical engineering solutions, to ‘’Smart Water Practices and Management’’, which employ big data and advanced analytics to create new business insights covering the full cycle of resource management and service delivery. To be successful, utilities should integrate all the major aspects of a big data platform; viz., integration, analytics, visualisation, development, workload optimisation, security and governance.

Integration envisages creation of a single platform managing the data, as siloed data will only generate fragmented insights. Analytics would provide actionable information and visualisation and would function as an effective, understandable decision support system.  Development tools will enhance the analytical and visualisation engines and the workload optimisation focuses on efficient processing and storage of data. Data security and governance are essential to protect sensitive data, which is critical for water utilities and the public sector.

Key barriers and drivers

In spite of the visible advantages of embracing smart water practices, the barriers are many, most critical being the lack of incentives for change. There is no necessity or compulsion as the utilities are seldom asked to be accountable for performance or return on investment. Massive amount of data generated by the sector is in inter- and intra-utility siloes and is not amenable for meaningful integrated analysis. In the absence of full cost recovery and ring fenced financing, utilities are also starved of funds to invest in smart, intelligent systems. In a nutshell, efficiency improvement is not a compulsive survival proposition for the utilities, managers and owners.

KWA- JICA project, Kozhikode, Kerala, India

In India, efforts have been initiated in benchmarking urban water utilities in the early 2000s by the Ministry of Urban Development in collaboration with WSP-SA. However, most utilities have not been able to provide reliable data on their network performance. In several utilities, even the length of the distribution system or the sewerage network is not known. In other cases, critical data is not maintained at a centralised level and utilities found it difficult to aggregate the data. There is no systematic process for recording information on the number of pipe breaks or sewer blockages in several utilities. Hence, fundamental to the migration to smart practices is to improve the data systems. In other words, while there is an immediate need to move ahead with smart practices, one has to set the basics right by improving the data systems.

Secondly, no amount of change is possible unless the utilities are made accountable to the tax payers and customers. As the sector is basking in an unchallenged comfort zone, given the prevailing incentive-dis-incentive structure one cannot normally expect a utility to change on its own. Therefore to encourage utilities to adopt big data analytics requires a considerable amount of preparation and the creation of an enabling environment by the Government. Though the customers clamour for better services, their voice is rarely heard. Hence, yet another key ‘’driver’’ is to make utilities accountable for performance with the clear message that subsidies, if any, are to support the poor and not to cover the inefficiencies of service providers.

Thirdly, as a corollary to the accountability mechanism, there is urgent need for an independent regulator for every state to benchmark performance, cost efficiency, tariff setting and service delivery standards. The regulator will also make the Governments accountable by ring fencing budgets to cover a timely release of subsidies. The utilities have to submit to the regulator clear road maps to achieve agreed indicators and efficiency standards.

Fourthly, one has to set right the quality of primary measurements and data. Many utilities have already installed smart systems. They generate data from supervisory control and data acquisition (SCADA) systems, flow statistics, online monitoring, water quality analysis, dissolved oxygen (DO) measurements and computerised maintenance management systems (CMMS), public grievance redressal systems etc. However, the data generated is not being processed effectively for decision making. Generating quality data is critical for analytical accuracy. Access to data and data integrity are the manifestation of organisational conviction, accountability and culture. If sensors are not properly calibrated, cleaned or maintained, generated data sets drive us to disastrous findings.

Fifthly, utility mindset should change from hardware solutions to management practices. Any discussion on performance normally ends up as a demand for huge investments to replace the entire aging capital stock, including pumping and distribution systems and other facilities. Utilities seldom agree that significant efficiency improvement is possible with existing systems. Performance is part of a management and organisational culture and not investments per se. Utilities are to be handheld to make the migration possible, adopting a mix of short and medium term investment strategies of asset management, maintenance, modernisation and analytics.

Last, but not least, successfully leveraging the possibilities of data analytics requires conscientious people and process re-engineering. Make the employees an integral part through incentives and motivation to stimulate the migration to smart water practices and management. The analytical results shall also have to find their way in the form of results based process re-engineering. Data analytics, in order to be successful in ushering in a new era of water reforms, needs to be offered as an integrated solution package and rolled out by accountable and motivated managers. 


At IRC we have strong opinions and we value honest and frank discussion, so you won't be surprised to hear that not all the opinions on this site represent our official policy.