Forecasting Kaveri River Water Quality using Time series models

  • February 28, 2024
  • Blog
  • 0 Comments

People around the globe, are affected by poor water quality. Potable water scarcity is exacerbated due to poor water quality. It has been observed in some countries that there is a direct correlation between water quality and economic growth 1. Therefore it is imperative to study and forecast water quality so that timely action could be taken and policy could …

Continue Reading

Secure Multi-Party Computation

  • December 19, 2023
  • Blog
  • 0 Comments

Have you ever wanted to query an online application but felt sceptical about sharing your personal data? How can confidential data be given to a third party for analysis without worrying about its privacy implications? How do multiple competing organisations collaborate on a joint task without revealing their private data? Private data is not limited to personally identifiable information and …

Continue Reading

Data Marketplace : A system design perspective

  • December 11, 2023
  • Blog
  • 0 Comments

A data exchange platform such as IUDX or ADeX curates datasets from various vendors based on how reliable, relevant and accurate the datasets are. A byproduct of this process is that the data that’s hosted on the platform ends up being quite valuable. One of the ways to realise this value is to build a marketplace. This encourages vendors to …

Continue Reading

Profiling Java Applications using Async Profiler and Flame Graphs

Introduction A key requirement of a data exchange is good platform performance. Deciding what is good performance is usually determined by the data characteristics and query statistics. Serving a hundred small JSON documents per second may be considered slow, while serving a hundred 5-minute-long video files per second could be considered as excellent performance. There are various metrics which can …

Continue Reading

Differential Privacy for Smart Cities

Many of the keystone applications running a “smart” city will require data sharing between parties with heterogenous interests. As an example, let’s imagine an “intelligent” transport management system (ITMS) that might optimise end-to-end transit times, measure and mitigate air pollution, manage city-wide transport carbon emissions, minimize traffic congestion and signal waiting times, support planning and building new infrastructure, etc. This …

Continue Reading

Are you holding onto a valuable asset – a High Value Dataset?

  • December 23, 2022
  • Blog
  • 0 Comments

The world has become increasingly digital and a very high volume of data is being generated and consumed of late. It is therefore important that the data owners are able to identify whether the dataset they have is a High Value Dataset (HVD) or not. This blog is therefore an attempt to provide a template for distinguishing an HVD. The Definition: The datasets …

Continue Reading

Defining Metrics to Assess the Quality of Data

  • December 20, 2022
  • Blog
  • 0 Comments

In the previous post, we explored the reasons that a Data Quality Assessment (DQA) framework is important for a data exchange platform such as IUDX. We looked at how such a DQA framework/tool would be modularly designed and positioned in the existing IUDX architecture, and the benefits to both upstream and downstream stakeholders. In this post, we will discuss the …

Continue Reading

IUDX for e-Governance

One of the fundamental responsibilities of the government is to provide services and benefits to its citizens. These services and benefits may be welfare-centric schemes like Direct Benefit Transfer (DBT), regulatory services abiding by the law, statutory services, and consumer utility services such as water supply, electricity, and other basic amenities. In FY 22-23, there has been a total of …

Continue Reading

The Importance of Data Quality for a Data Exchange Platform

The global economy is ever-increasingly reliant on the data that drives it and the ecosystem surrounding data is rapidly growing in size and scope. Data is used to frame and enact policies, drive better governance, run businesses and perform data science. India Urban Data Exchange (IUDX) forms part of this data ecosystem by facilitating seamless exchange of data and in …

Continue Reading